-- Logs begin at Sun 2022-09-11 04:50:15 UTC, end at Sun 2022-09-11 06:22:07 UTC. -- Sep 11 05:15:16.143783 dv-cinder-856921-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 11 05:15:16.153914 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 11 05:15:16.155918 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 11 05:15:16 2022] *** Sep 11 05:15:16.155918 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 11 05:15:16.155918 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 11 05:15:16.155918 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: nodename: dv-cinder-856921-1-iscsi Sep 11 05:15:16.155918 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: machine: x86_64 Sep 11 05:15:16.155918 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: clock source: unix Sep 11 05:15:16.155918 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: pcre jit disabled Sep 11 05:15:16.155918 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: detected number of CPU cores: 4 Sep 11 05:15:16.155918 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: current working directory: / Sep 11 05:15:16.155918 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: detected binary path: /usr/bin/uwsgi-core Sep 11 05:15:16.155918 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: your processes number limit is 47680 Sep 11 05:15:16.156650 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: your memory page size is 4096 bytes Sep 11 05:15:16.156650 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: detected max file descriptor number: 2048 Sep 11 05:15:16.156650 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: lock engine: pthread robust mutexes Sep 11 05:15:16.156650 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: thunder lock: enabled Sep 11 05:15:16.156650 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 11 05:15:16.156650 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 11 05:15:16.195417 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: Python main interpreter initialized at 0x55cb6087a9c0 Sep 11 05:15:16.195417 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: python threads support enabled Sep 11 05:15:16.195417 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: your server socket listen backlog is limited to 100 connections Sep 11 05:15:16.195417 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: your mercy for graceful operations on workers is 90 seconds Sep 11 05:15:16.195787 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: mapped 403077 bytes (393 KB) for 2 cores Sep 11 05:15:16.195869 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: *** Operational MODE: preforking *** Sep 11 05:15:16.196132 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: *** uWSGI is running in multiple interpreter mode *** Sep 11 05:15:16.196132 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: spawned uWSGI master process (pid: 81429) Sep 11 05:15:16.196331 dv-cinder-856921-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 11 05:15:16.197156 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: spawned uWSGI worker 1 (pid: 81444, cores: 1) Sep 11 05:15:16.197252 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: spawned uWSGI worker 2 (pid: 81445, cores: 1) Sep 11 05:15:16.198168 dv-cinder-856921-1-iscsi devstack@keystone.service[81429]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 11 05:15:20.488209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 05:15:20.488209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: from cryptography.utils import int_from_bytes Sep 11 05:15:20.489004 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 05:15:20.489004 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: from cryptography.utils import int_from_bytes Sep 11 05:15:20.489429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 05:15:20.489429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: from cryptography.utils import int_from_bytes Sep 11 05:15:20.489542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 05:15:20.489542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: from cryptography.utils import int_from_bytes Sep 11 05:15:20.539392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 05:15:20.539570 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 05:15:20.539667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 05:15:20.539756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 05:15:20.539837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] command line args: [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 05:15:20.539916 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] command line args: [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 05:15:20.540050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 05:15:20.540144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 05:15:20.540219 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ================================================================================ {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 05:15:20.540291 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ================================================================================ {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 05:15:20.540482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] admin_token = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.540563 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] admin_token = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.540733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] config_dir = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.540822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] config_dir = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.540928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.541016 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.541096 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] config_source = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.541174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] config_source = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.541257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] debug = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.541337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] debug = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.541462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.541768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.541910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.542017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.542115 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.542216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.542321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.542416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.542519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.542618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.542731 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.542834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.542937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_config_append = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.543034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_config_append = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.543140 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.543243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.543345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_dir = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.543442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_dir = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.543548 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_file = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.543650 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_file = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.543756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.543844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.543963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.544088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.544163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.544281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.544370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.544453 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.544581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.544659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.544784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.544870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.544944 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.545063 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.545138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.545254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.545342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.545423 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.545548 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.545632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.545749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.545826 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.545979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.546067 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.546228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.546318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.546475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.546604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.546749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.546881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.547237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.547319 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.547449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] publish_errors = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.547526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] publish_errors = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.547643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.547718 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.547854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.547942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.548058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.548138 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.548310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.548409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.548545 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.548624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.548779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] standard_threads = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.548858 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] standard_threads = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.549010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.549098 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.549252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.549332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.549433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.549510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.549607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] use_journal = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.549737 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] use_journal = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.549813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] use_json = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.549884 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] use_json = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.549961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] use_stderr = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.550039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] use_stderr = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.550117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] use_syslog = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.550196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] use_syslog = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.550322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.550407 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 05:15:20.550578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.550655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.550812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.550898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.551071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.551164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.551310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.551390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.551560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.551651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.551800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.551879 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.552071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.552158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.552342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.552468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.552593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.552801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.552880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.553084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.553162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.553326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.553406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.553579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.553659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.553819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.553908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.554071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.554151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.554323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.554410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.554567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.554652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.554806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.554885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.555064 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.555143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.555310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.555466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.555547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.555621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.555776 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.555855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.556053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.556148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.556319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.556399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.556572 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.556656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.556811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.556898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.557044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.557123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.557278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.557357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.557512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.557605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.557749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.557829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.557998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.558076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.558233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.558320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.558467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.558544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.558698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.558777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.558946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.559041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.559190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.559267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.559428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.559508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.559674 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.559761 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.559905 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.559989 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.560165 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.560243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.560414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.560507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.560618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.560704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.560849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.560933 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.561076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.561233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.561307 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.561379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.561532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.561613 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.561779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.561865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.562008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.562088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.562245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.562322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.562474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.562564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.562710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.562791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.562945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.563030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.563182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.563272 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.563410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.563478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.563649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.563717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.563888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.563958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.564274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.564369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.564369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.564531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.564607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.564688 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.564842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.564971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.565085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.565180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.565359 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.565490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.565615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.565740 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.565874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.566010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.566144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.566269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.566406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.566545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.566676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.566805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.566950 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.567082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.567218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.567350 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.567471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.567595 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.567718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.567843 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.568547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.568634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.568791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.568926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.568926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.569164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.569302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.569302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.569536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.569660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.569787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.569912 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.570049 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.570223 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.570354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.570483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.570618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.570750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.570885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.571011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.571143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.571276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.571409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.571536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.571669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.571796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.571921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.572075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.572200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.572324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.572469 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.572597 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.572722 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.572848 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.572979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.573113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.573248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.573375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.573501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.573627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.573753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.573878 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.574080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.574215 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.574354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.574481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.574607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.574741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.574876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.575004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.575132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.575265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.575393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.575520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.575646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.575778 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.575903 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.576052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.576196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.576339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.576467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.576602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.576729 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.576855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.576980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.577106 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.577231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.577368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.577497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.577624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.577750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.577885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.external = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.578038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.578175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.578302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.578439 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.578575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.external = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.578703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.578829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.578961 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.password = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.579089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.579224 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] auth.token = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.579352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.579478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.579615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.password = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.579742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.579868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] auth.token = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.580044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.580171 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.580307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.580436 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.580563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.580691 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.580828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.580956 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.581084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.581210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.581336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.581473 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.581608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.581736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.581863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.581990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.582116 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.582244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.582244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.582492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.582619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.582747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.582873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.583007 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.583157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.583292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.583418 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.583673 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.584302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.584438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.584581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.584712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.584841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.584971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.585105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.585233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.585368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.585502 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.585642 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.585772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.585901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.586035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.586165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.586295 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.586425 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.586555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.586684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.586822 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.586953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.587083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.587212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.587342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.587481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.587621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.587752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.587883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.588063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.588190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.588319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.588457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.588585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.588585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.588827 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.588827 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.589064 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.589064 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.589321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.589447 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.589579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.589707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.589836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.590045 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.590186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.590310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.590439 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.590566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.590693 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.590829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.590957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.591093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.591220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.591346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.591472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.591606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.591733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.591860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.591987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.592131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.592260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.592402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.592531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.592657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.592796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.592931 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.593201 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.593329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.593456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.593686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.593813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.593942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.594082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.594211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.594346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.594474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.594602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.594739 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.594877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.595005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.595141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.595269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.595395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.595530 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.595657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.595783 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.595910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.596068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.596200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.596339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.596477 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.596605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.596773 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.596902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.597032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.597172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.597302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.597428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.597555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.597683 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.597810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.597946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.598073 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.598200 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.598340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.598468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.598596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.598731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.598892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.599020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.599183 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.599312 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.599440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.599579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.599706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.599831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.599955 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.600131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.600270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.600270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.600527 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.600655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.600782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.600908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.601036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.601171 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.601301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.601437 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.601566 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.601705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.601883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.602053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.602186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.602314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.602443 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.602571 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.602699 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.602833 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.602962 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.603089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.603216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.603342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.603470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.603605 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.603736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.603873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.604019 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.604147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.604276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.604415 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.604545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.604679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.604808 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.604936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.605062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.605197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.605325 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.605452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.605578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.605718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.605849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.605986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.606141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.606278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.606409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.606536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.606661 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.606794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.606922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.607047 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.607171 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.607296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.607432 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.607566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.607693 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.607818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.607944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.608110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.608237 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.608374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.608503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.608630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.608759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.608888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.609015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.609151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.609289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.609416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.609543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.609670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.609798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.609933 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.610062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.610190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.610317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.610444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.610571 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.610705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.610834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.610962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.611099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.611226 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.611226 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.611474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.611603 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.611730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.611730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.611970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.611970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.612233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.612364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.612493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.612623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.612762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.612900 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.613032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.613163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.613292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.613422 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.613551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.613689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.613819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.613949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.614079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.614209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.614338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.614476 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.614620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.614782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.614893 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.615068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.615195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.615397 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.615529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.615660 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] role.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.615790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.615920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] role.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.616104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] role.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.616267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.616394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] role.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.616524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.616662 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.616790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.616960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.617099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.617229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.617357 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.617486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.617614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.617741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.617878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.618012 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.618141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.618269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.618406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.618533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.618670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.618798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.618926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.619054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.619182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.619311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.619446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.619575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.619704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.619831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.619831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.620107 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.620107 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.620363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.620491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.620618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.620746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.620880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.621009 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.621138 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.621266 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.621394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.621521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.621656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.621795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.621924 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.622050 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.622178 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.622305 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.622305 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.622554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.622681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.622808 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.622935 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.623061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.623196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.623326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.623326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.623578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.623578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.623818 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.623953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.624121 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.624248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.624375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.624502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.624629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.624765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.624895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.625021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.625148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.625284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.625411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.625547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.625675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.625802 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.625932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.626058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.626190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.626324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.626469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.626650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.626803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.626934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.627062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.627211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.627340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.627469 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.627596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.627723 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.627851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.627985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.628397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.628479 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.628649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.628774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.628899 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.629036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.629176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.629321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.629450 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.629575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.629702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.629838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.connection = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.629974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.630100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.630235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.630361 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.631028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.connection = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.631155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.631281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.631406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.631550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.631677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.631804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.631931 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.632104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.632236 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.632374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.632503 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.632631 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.632758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.632887 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.633015 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.633152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.633281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.633417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.633545 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.633672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.633800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.633936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.634065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.634195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.634322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.634322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.634563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.634700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.634829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.634958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.635086 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.635225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81445) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 05:15:20.635362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.635362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.635362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 05:15:20.635362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81444) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 05:15:20.658017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.658225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.658375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.658513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.658642 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.658772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.667215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.667460 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.695702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.695898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.696061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.696193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.748891 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.749657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.798638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.798897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.799088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.799088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.955132 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.955324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.955413 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.955493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.955597 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.955678 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.955788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.955875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.956022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.956124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.956254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.956338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.979612 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.979840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.979928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.980090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.980177 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.980317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.980402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.980540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.980637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.980760 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.980845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.980975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.981057 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.981190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.992648 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:20.992796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.005632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.005850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.005964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.006104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.006272 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.006370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.006515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.006606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.006735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.006826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.006941 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81444) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.007031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.007161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.007380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81445) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 05:15:21.024416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.024637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 05:15:21.024724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.024883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 05:15:21.024966 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.025174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 05:15:21.025256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.025410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 05:15:21.025493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.025685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.025777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.025949 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.026032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.026206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.026287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.026466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.035752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.035935 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.036059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.036242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.036325 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.036517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.036602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.036903 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.043729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.043820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.046641 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.046727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.046929 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.047013 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.047214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.047294 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.047495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.047578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.047776 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.047861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.048060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.048158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.048332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.048416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.048607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.048689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.048874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.048957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.049145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.049228 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.049412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.049494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.072407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.072620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.073854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.074091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.074176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.074382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.078729 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.079018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.079101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.079295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.079392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.079601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.086411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.086509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.086696 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.086780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.092270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.092399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.092564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.092701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.092851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.092998 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.093140 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.093292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.093422 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.093607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.093703 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.093866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.093957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.094146 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.094228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.094426 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.109609 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.109857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.109943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.110140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.113708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.114006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.117520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.117818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.117902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.118192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.122009 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.122422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.125117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.125565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.128533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.128832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.129017 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.129102 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.129322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.129405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.129612 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.129697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.129903 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.129988 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.130186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.131065 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.145897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.146349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.146511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.146805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.146898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.147075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.147161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.147346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.153805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.154112 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.155303 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.155627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.156823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.157103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.157186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.157438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.159805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.160117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.160201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.160400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.160484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.160682 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.160775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.160974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.161065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.161372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.172228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.172494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.172579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.172772 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.172866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.173053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.173137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.173338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.173421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.173704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.173795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.173984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.174069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.174291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.189687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.190113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.192621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.193630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.196300 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.196528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.197732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.198017 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.199181 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.199464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.199551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.199769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.199899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.200192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.206113 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.206545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.207560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.208017 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.210554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.210950 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.211098 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.211237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.211500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.211596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.211790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.212108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.219226 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.219519 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.219801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.219884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.220216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.220506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.227120 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81445) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.227410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.227683 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.227958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.228057 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81444) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 05:15:21.228265 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.228347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.228538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.228624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.228812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.228895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.229087 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.229180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.229365 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.229456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.229645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.229737 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.229915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.229998 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.230186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.230269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.230451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.230536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.230727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81445) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.230810 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.231005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.231275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.231565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81444) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 05:15:21.265225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55cb6087a9c0 pid: 81445 (default app) Sep 11 05:15:21.266273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55cb6087a9c0 pid: 81444 (default app) Sep 11 05:15:21.268412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04470f-0789-4281-b465-a2243b96eda8 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:21.268699 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04470f-0789-4281-b465-a2243b96eda8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:21.268969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04470f-0789-4281-b465-a2243b96eda8 None None] PATH_INFO: `/v3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:21.273309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Sun Sep 11 05:15:21 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:28.037825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd8500-11a6-454d-aafa-10ba59281aae None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:28.038371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd8500-11a6-454d-aafa-10ba59281aae None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:28.038371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd8500-11a6-454d-aafa-10ba59281aae None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:28.042657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:15:28.047844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f933b-76b8-4602-a611-15b41ef713fb None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:28.048385 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f933b-76b8-4602-a611-15b41ef713fb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:28.048482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f933b-76b8-4602-a611-15b41ef713fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:28.117765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: INFO dbcounter [None req-da2f933b-76b8-4602-a611-15b41ef713fb None None] Registered counter for database keystone Sep 11 05:15:28.137205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG oslo_db.sqlalchemy.engines [None req-da2f933b-76b8-4602-a611-15b41ef713fb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81445) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 05:15:28.143332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writer thread running {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 05:15:28.321691 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG passlib.handlers.bcrypt [None req-da2f933b-76b8-4602-a611-15b41ef713fb None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81445) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 11 05:15:28.322134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG passlib.handlers.bcrypt [None req-da2f933b-76b8-4602-a611-15b41ef713fb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81445) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 11 05:15:28.383839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-da2f933b-76b8-4602-a611-15b41ef713fb None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:28.385426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-da2f933b-76b8-4602-a611-15b41ef713fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:28.516346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 11 05:15:28.818595 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd8500-11a6-454d-aafa-10ba59281aae None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:28.888171 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: INFO dbcounter [None req-eecd8500-11a6-454d-aafa-10ba59281aae None None] Registered counter for database keystone Sep 11 05:15:28.909147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG oslo_db.sqlalchemy.engines [None req-eecd8500-11a6-454d-aafa-10ba59281aae None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81444) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 05:15:28.915919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writer thread running {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 05:15:29.033420 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd8500-11a6-454d-aafa-10ba59281aae None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:29.038164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4a272-a218-49cd-b005-7ea68b8f61f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:29.038795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4a272-a218-49cd-b005-7ea68b8f61f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:29.039078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4a272-a218-49cd-b005-7ea68b8f61f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:29.039353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4a272-a218-49cd-b005-7ea68b8f61f7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:29.047499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4a272-a218-49cd-b005-7ea68b8f61f7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:29.098268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c2d4a272-a218-49cd-b005-7ea68b8f61f7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 11 05:15:29.098618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4a272-a218-49cd-b005-7ea68b8f61f7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:29.107900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Sun Sep 11 05:15:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 05:15:29.113870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f933b-76b8-4602-a611-15b41ef713fb None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:29.135373 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f933b-76b8-4602-a611-15b41ef713fb None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:29.139753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315658b-b582-4f4f-bf6c-7b88d2706355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:29.140892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315658b-b582-4f4f-bf6c-7b88d2706355 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:29.141181 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315658b-b582-4f4f-bf6c-7b88d2706355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:29.141455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315658b-b582-4f4f-bf6c-7b88d2706355 admin admin] PATH_INFO: `/v3/projects/4c05020216314cd9bc11af667152fcc9` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:29.143033 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315658b-b582-4f4f-bf6c-7b88d2706355 admin admin] RBAC: Authorizing `identity:get_project(project_id=4c05020216314cd9bc11af667152fcc9)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:29.195858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8315658b-b582-4f4f-bf6c-7b88d2706355 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 11 05:15:29.196229 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315658b-b582-4f4f-bf6c-7b88d2706355 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:29.200743 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Sun Sep 11 05:15:29 2022] GET /identity/v3/projects/4c05020216314cd9bc11af667152fcc9 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:15:31.317009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cecad-1464-4a59-8978-4f52281a0bb7 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:31.317546 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cecad-1464-4a59-8978-4f52281a0bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:31.317546 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cecad-1464-4a59-8978-4f52281a0bb7 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:31.323211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:15:31.328818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed21e37-4b9b-42ec-9847-b391f3bf2644 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:31.329228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed21e37-4b9b-42ec-9847-b391f3bf2644 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:31.329368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed21e37-4b9b-42ec-9847-b391f3bf2644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:31.350698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-eed21e37-4b9b-42ec-9847-b391f3bf2644 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:31.352241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-eed21e37-4b9b-42ec-9847-b391f3bf2644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:31.359895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 11 05:15:31.625359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cecad-1464-4a59-8978-4f52281a0bb7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:31.637566 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cecad-1464-4a59-8978-4f52281a0bb7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:31.641761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b535cdc-8384-4786-b8e1-5ddb59b3a423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:31.642363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b535cdc-8384-4786-b8e1-5ddb59b3a423 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:31.642646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b535cdc-8384-4786-b8e1-5ddb59b3a423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:31.642920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b535cdc-8384-4786-b8e1-5ddb59b3a423 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:31.650981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b535cdc-8384-4786-b8e1-5ddb59b3a423 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:31.652797 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b535cdc-8384-4786-b8e1-5ddb59b3a423 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:31.661418 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Sun Sep 11 05:15:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 05:15:31.667150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed21e37-4b9b-42ec-9847-b391f3bf2644 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:31.676312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed21e37-4b9b-42ec-9847-b391f3bf2644 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:31.680825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b603b-50b6-45a4-ae23-f3578320fb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:31.681453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b603b-50b6-45a4-ae23-f3578320fb3f admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:31.681739 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b603b-50b6-45a4-ae23-f3578320fb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:31.682017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b603b-50b6-45a4-ae23-f3578320fb3f admin admin] PATH_INFO: `/v3/users/f615cddbb3154c329188ec780374eff5` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:31.683322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b603b-50b6-45a4-ae23-f3578320fb3f admin admin] RBAC: Authorizing `identity:get_user(user_id=f615cddbb3154c329188ec780374eff5)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:31.685196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b603b-50b6-45a4-ae23-f3578320fb3f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:31.686710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Sun Sep 11 05:15:31 2022] GET /identity/v3/users/f615cddbb3154c329188ec780374eff5 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:33.899264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee58df-319e-46c8-8754-cf48de8a682d None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:33.899785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee58df-319e-46c8-8754-cf48de8a682d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:33.899886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee58df-319e-46c8-8754-cf48de8a682d None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:33.901167 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:33.906260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a0642-4d35-42d3-8764-289dce7f3817 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:33.906675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a0642-4d35-42d3-8764-289dce7f3817 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:33.906823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a0642-4d35-42d3-8764-289dce7f3817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:33.924793 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-110a0642-4d35-42d3-8764-289dce7f3817 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:33.925983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-110a0642-4d35-42d3-8764-289dce7f3817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:33.927459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d86780-62c4-4d21-bc81-83323ed75c2c None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:33.927747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d86780-62c4-4d21-bc81-83323ed75c2c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:33.928056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d86780-62c4-4d21-bc81-83323ed75c2c None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:33.929395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:15:33.934635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f61d62-5c36-4116-8ce1-9bb272dfc66d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:33.934924 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f61d62-5c36-4116-8ce1-9bb272dfc66d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:33.935199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f61d62-5c36-4116-8ce1-9bb272dfc66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:33.977554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG passlib.handlers.bcrypt [None req-12f61d62-5c36-4116-8ce1-9bb272dfc66d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81444) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 11 05:15:33.978261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG passlib.handlers.bcrypt [None req-12f61d62-5c36-4116-8ce1-9bb272dfc66d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81444) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 11 05:15:34.010791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:15:34.028243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-12f61d62-5c36-4116-8ce1-9bb272dfc66d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:34.029840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-12f61d62-5c36-4116-8ce1-9bb272dfc66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:34.126862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:15:34.282466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a0642-4d35-42d3-8764-289dce7f3817 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:34.307969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:34.389245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a0642-4d35-42d3-8764-289dce7f3817 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:34.398328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f61d62-5c36-4116-8ce1-9bb272dfc66d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:34.424478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11babd-eacb-4c76-8208-1d5cac0f365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:34.425121 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11babd-eacb-4c76-8208-1d5cac0f365e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:34.425405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11babd-eacb-4c76-8208-1d5cac0f365e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:34.425687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11babd-eacb-4c76-8208-1d5cac0f365e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:34.436785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11babd-eacb-4c76-8208-1d5cac0f365e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-2c11babd-eacb-4c76-8208-1d5cac0f365e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:15:34.445672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:34.446800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:34.447886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:34.447886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:34.447886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:34.447886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:34.447886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:15:34.447886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:15:34.447886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:15:34.447886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:15:34.447886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:15:34.447886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:15:34.447886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:15:34.447886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:15:34.447886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:15:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:15:34.452525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11babd-eacb-4c76-8208-1d5cac0f365e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:34.503628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f61d62-5c36-4116-8ce1-9bb272dfc66d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:34.505956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:34.536771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:34.542218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999072c5-60b9-4909-b13c-fd054703553d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:34.543083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999072c5-60b9-4909-b13c-fd054703553d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:34.543364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999072c5-60b9-4909-b13c-fd054703553d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:34.543635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999072c5-60b9-4909-b13c-fd054703553d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:34.556291 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11babd-eacb-4c76-8208-1d5cac0f365e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:34.590307 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e1087-b639-40fd-9e22-adb2cd8088a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:34.590910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e1087-b639-40fd-9e22-adb2cd8088a0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:34.591189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e1087-b639-40fd-9e22-adb2cd8088a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:34.591461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e1087-b639-40fd-9e22-adb2cd8088a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:34.592157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e1087-b639-40fd-9e22-adb2cd8088a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:34.592457 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e1087-b639-40fd-9e22-adb2cd8088a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:34.593849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e1087-b639-40fd-9e22-adb2cd8088a0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:34.605777 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:15:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:15:34.610099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999072c5-60b9-4909-b13c-fd054703553d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:34.611444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e1087-b639-40fd-9e22-adb2cd8088a0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:34.612036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999072c5-60b9-4909-b13c-fd054703553d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:34.710598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e1087-b639-40fd-9e22-adb2cd8088a0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:34.717525 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:15:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 05:15:34.723337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999072c5-60b9-4909-b13c-fd054703553d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:34.726649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:34.745412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889f512-ff21-4969-b53f-6b733ef8d114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:34.746057 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889f512-ff21-4969-b53f-6b733ef8d114 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:34.746342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889f512-ff21-4969-b53f-6b733ef8d114 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:34.746616 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889f512-ff21-4969-b53f-6b733ef8d114 None admin] PATH_INFO: `/v3/users/f615cddbb3154c329188ec780374eff5` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:34.747756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889f512-ff21-4969-b53f-6b733ef8d114 None admin] RBAC: Authorizing `identity:get_user(user_id=f615cddbb3154c329188ec780374eff5)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:34.749259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889f512-ff21-4969-b53f-6b733ef8d114 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:34.750820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:15:34 2022] GET /identity/v3/users/f615cddbb3154c329188ec780374eff5 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:15:34.756589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889f512-ff21-4969-b53f-6b733ef8d114 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:34.770359 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:34.823680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999072c5-60b9-4909-b13c-fd054703553d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:34.856954 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65735662-67b5-4c7b-b6a3-cb0ae3634950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:34.857548 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889f512-ff21-4969-b53f-6b733ef8d114 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:34.857808 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65735662-67b5-4c7b-b6a3-cb0ae3634950 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:34.858089 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65735662-67b5-4c7b-b6a3-cb0ae3634950 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:34.858366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65735662-67b5-4c7b-b6a3-cb0ae3634950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:34.876582 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65735662-67b5-4c7b-b6a3-cb0ae3634950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:34.878031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65735662-67b5-4c7b-b6a3-cb0ae3634950 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:34.893185 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036fad04-3b0a-4f20-afcf-8274233ec733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:34.893815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fad04-3b0a-4f20-afcf-8274233ec733 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:34.894100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fad04-3b0a-4f20-afcf-8274233ec733 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:34.894373 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fad04-3b0a-4f20-afcf-8274233ec733 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:34.895484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036fad04-3b0a-4f20-afcf-8274233ec733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-65735662-67b5-4c7b-b6a3-cb0ae3634950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:15:34.896061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:34.897044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:15:34.897982 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:15:34.897982 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:15:34.897982 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:15:34.898137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036fad04-3b0a-4f20-afcf-8274233ec733 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:34.898210 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:15:34.899194 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:15:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:15:34.903388 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65735662-67b5-4c7b-b6a3-cb0ae3634950 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:34.904600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036fad04-3b0a-4f20-afcf-8274233ec733 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:34.961167 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:34.985310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:35.005608 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036fad04-3b0a-4f20-afcf-8274233ec733 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:35.006085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65735662-67b5-4c7b-b6a3-cb0ae3634950 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:35.039620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c7c00-f767-4303-a049-8b47535a49cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:35.039786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53538c5f-a9d0-4a78-9583-94a62c190409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:35.040260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c7c00-f767-4303-a049-8b47535a49cb None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:35.040379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53538c5f-a9d0-4a78-9583-94a62c190409 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:35.040542 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c7c00-f767-4303-a049-8b47535a49cb None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:35.040630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53538c5f-a9d0-4a78-9583-94a62c190409 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:35.040815 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c7c00-f767-4303-a049-8b47535a49cb None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:35.040911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53538c5f-a9d0-4a78-9583-94a62c190409 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:35.041457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c7c00-f767-4303-a049-8b47535a49cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:35.041557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53538c5f-a9d0-4a78-9583-94a62c190409 None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.domain.id=default, user.id=f615cddbb3154c329188ec780374eff5) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:35.041759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c7c00-f767-4303-a049-8b47535a49cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:35.041845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53538c5f-a9d0-4a78-9583-94a62c190409 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:35.043141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c7c00-f767-4303-a049-8b47535a49cb None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:35.043256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53538c5f-a9d0-4a78-9583-94a62c190409 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:35.049889 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 11/20] 10.222.0.33 () {56 vars in 1322 bytes} [Sun Sep 11 05:15:34 2022] GET /identity/v3/role_assignments?user.id=f615cddbb3154c329188ec780374eff5&scope.domain.id=default&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:35.061581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 10/21] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:15:37.904657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4df43d-224c-4e4e-a4fc-4cdea873f071 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:37.905716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4df43d-224c-4e4e-a4fc-4cdea873f071 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:37.906226 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4df43d-224c-4e4e-a4fc-4cdea873f071 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:37.907788 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 12/22] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:37.914098 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f288c0-92f9-4e2c-9355-e481de24642b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:37.914595 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f288c0-92f9-4e2c-9355-e481de24642b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:37.914670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f288c0-92f9-4e2c-9355-e481de24642b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:37.936988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-12f288c0-92f9-4e2c-9355-e481de24642b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:37.938190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-12f288c0-92f9-4e2c-9355-e481de24642b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:37.953231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f227c6b-b447-4465-adcd-4319c3834478 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:37.953866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f227c6b-b447-4465-adcd-4319c3834478 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:37.954354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f227c6b-b447-4465-adcd-4319c3834478 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:37.955907 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 13/23] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:15:37.961454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d31d87-8cd7-4fe3-8054-dac155118b1b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:37.962002 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d31d87-8cd7-4fe3-8054-dac155118b1b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:37.962501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d31d87-8cd7-4fe3-8054-dac155118b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:37.984019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-45d31d87-8cd7-4fe3-8054-dac155118b1b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:37.985610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-45d31d87-8cd7-4fe3-8054-dac155118b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:38.044477 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 11/24] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:15:38.047251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c99f5b-6110-48ba-8a0d-6fe4a1ab58d4 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:38.049657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c99f5b-6110-48ba-8a0d-6fe4a1ab58d4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:38.049945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c99f5b-6110-48ba-8a0d-6fe4a1ab58d4 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:38.051250 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 12/25] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:38.056932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d13f4-d883-4f78-bcc1-0a323747d4da None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:38.057142 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d13f4-d883-4f78-bcc1-0a323747d4da None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:38.057579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d13f4-d883-4f78-bcc1-0a323747d4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:38.071360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 14/26] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:15:38.083927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-526d13f4-d883-4f78-bcc1-0a323747d4da None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:38.085516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-526d13f4-d883-4f78-bcc1-0a323747d4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:38.160266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:38.195854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 13/27] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:15:38.222054 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6fa65-66c3-4800-a230-b42353631c0b None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:38.222346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6fa65-66c3-4800-a230-b42353631c0b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:38.222624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6fa65-66c3-4800-a230-b42353631c0b None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:38.224180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 15/28] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:15:38.230180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b1254-4d13-4c0e-b101-299c90391d2f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:38.230657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b1254-4d13-4c0e-b101-299c90391d2f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:38.230949 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b1254-4d13-4c0e-b101-299c90391d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:38.255061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-413b1254-4d13-4c0e-b101-299c90391d2f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:38.256291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-413b1254-4d13-4c0e-b101-299c90391d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:38.346463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 14/29] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:15:38.406051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6fa65-66c3-4800-a230-b42353631c0b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:38.425048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b1254-4d13-4c0e-b101-299c90391d2f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:38.540342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:38.590770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6fa65-66c3-4800-a230-b42353631c0b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:38.613270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b1254-4d13-4c0e-b101-299c90391d2f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:38.623732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:38.658911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a91dd-d806-4d4b-8307-f032daad337d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:38.659659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a91dd-d806-4d4b-8307-f032daad337d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:38.661708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a91dd-d806-4d4b-8307-f032daad337d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:38.662745 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a91dd-d806-4d4b-8307-f032daad337d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:38.680910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855779cb-1553-48cc-8e9a-f0c424bf6773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:38.681565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855779cb-1553-48cc-8e9a-f0c424bf6773 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:38.681850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855779cb-1553-48cc-8e9a-f0c424bf6773 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:38.682125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855779cb-1553-48cc-8e9a-f0c424bf6773 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:38.683342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a91dd-d806-4d4b-8307-f032daad337d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:38.684983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a91dd-d806-4d4b-8307-f032daad337d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:38.698160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855779cb-1553-48cc-8e9a-f0c424bf6773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-855779cb-1553-48cc-8e9a-f0c424bf6773 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:15:38.705543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:38.706379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:38.707159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:38.707159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:38.707159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:38.707159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:38.707159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:15:38.707159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:15:38.707159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:15:38.707159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:15:38.707159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:15:38.707159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:15:38.707159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:15:38.707159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-b79a91dd-d806-4d4b-8307-f032daad337d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:15:38.708631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:38.709497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:15:38.710306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:15:38.710306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:15:38.710306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:15:38.710468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 15/30] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:15:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:15:38.710468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855779cb-1553-48cc-8e9a-f0c424bf6773 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:38.710587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 16/31] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:15:38.710587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a91dd-d806-4d4b-8307-f032daad337d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:38.837951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a91dd-d806-4d4b-8307-f032daad337d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:38.847732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855779cb-1553-48cc-8e9a-f0c424bf6773 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:38.865846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:38.875714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ba770-52d5-4796-b500-aa512e862328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:38.876537 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ba770-52d5-4796-b500-aa512e862328 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:38.876979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ba770-52d5-4796-b500-aa512e862328 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:38.877395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ba770-52d5-4796-b500-aa512e862328 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:38.878452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ba770-52d5-4796-b500-aa512e862328 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:38.880126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ba770-52d5-4796-b500-aa512e862328 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:38.892270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333ffa3-ce30-4582-ba07-dce836b94e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:38.893183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333ffa3-ce30-4582-ba07-dce836b94e8c None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:38.893669 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333ffa3-ce30-4582-ba07-dce836b94e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:38.894436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333ffa3-ce30-4582-ba07-dce836b94e8c None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:38.895329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333ffa3-ce30-4582-ba07-dce836b94e8c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:38.897988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333ffa3-ce30-4582-ba07-dce836b94e8c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:39.016443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 17/32] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 11 05:15:38 2022] POST /identity/v3/roles => generated 219 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:15:39.017645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ba770-52d5-4796-b500-aa512e862328 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:39.029805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 16/33] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 11 05:15:38 2022] POST /identity/v3/roles => generated 225 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:15:39.031539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333ffa3-ce30-4582-ba07-dce836b94e8c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:39.068225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:39.168368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333ffa3-ce30-4582-ba07-dce836b94e8c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:39.214456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a0a0f-b6e0-4614-8e46-15d9e2e0020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:39.214671 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ba770-52d5-4796-b500-aa512e862328 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:39.215604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a0a0f-b6e0-4614-8e46-15d9e2e0020c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:39.216396 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a0a0f-b6e0-4614-8e46-15d9e2e0020c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:39.217895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a0a0f-b6e0-4614-8e46-15d9e2e0020c None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:39.218799 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a0a0f-b6e0-4614-8e46-15d9e2e0020c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:39.219311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a0a0f-b6e0-4614-8e46-15d9e2e0020c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:39.221137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a0a0f-b6e0-4614-8e46-15d9e2e0020c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:39.238260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 17/34] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:39.244375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a0a0f-b6e0-4614-8e46-15d9e2e0020c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:39.260511 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990df4b-b5be-440a-9a6e-faf761bca731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:39.261163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990df4b-b5be-440a-9a6e-faf761bca731 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:39.261448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990df4b-b5be-440a-9a6e-faf761bca731 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:39.261725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990df4b-b5be-440a-9a6e-faf761bca731 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:39.262434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990df4b-b5be-440a-9a6e-faf761bca731 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:39.263059 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990df4b-b5be-440a-9a6e-faf761bca731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:39.264747 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990df4b-b5be-440a-9a6e-faf761bca731 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:39.275559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 18/35] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:15:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:39.281212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990df4b-b5be-440a-9a6e-faf761bca731 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:39.340018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:39.364599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:39.387220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a0a0f-b6e0-4614-8e46-15d9e2e0020c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:39.403921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990df4b-b5be-440a-9a6e-faf761bca731 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:39.435317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da66bc-4bda-4e7d-9b61-1c334fe241f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:39.436404 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da66bc-4bda-4e7d-9b61-1c334fe241f5 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:39.436915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da66bc-4bda-4e7d-9b61-1c334fe241f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:39.437399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da66bc-4bda-4e7d-9b61-1c334fe241f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:39.438820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da66bc-4bda-4e7d-9b61-1c334fe241f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:39.441318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da66bc-4bda-4e7d-9b61-1c334fe241f5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:39.449509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2a6ee-a9df-47df-bd4a-fc7c1612c356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:39.450156 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2a6ee-a9df-47df-bd4a-fc7c1612c356 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:39.450732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2a6ee-a9df-47df-bd4a-fc7c1612c356 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:39.451008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2a6ee-a9df-47df-bd4a-fc7c1612c356 None admin] PATH_INFO: `/v3/users/f615cddbb3154c329188ec780374eff5` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:39.452311 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2a6ee-a9df-47df-bd4a-fc7c1612c356 None admin] RBAC: Authorizing `identity:get_user(user_id=f615cddbb3154c329188ec780374eff5)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:39.453813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2a6ee-a9df-47df-bd4a-fc7c1612c356 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:39.455627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 19/36] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:15:39 2022] GET /identity/v3/users/f615cddbb3154c329188ec780374eff5 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:15:39.461551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2a6ee-a9df-47df-bd4a-fc7c1612c356 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:39.480837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 18/37] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:15:39 2022] POST /identity/v3/projects => generated 301 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:15:39.596241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2a6ee-a9df-47df-bd4a-fc7c1612c356 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:39.618261 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:39.640468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cc5f0-7aa3-4de3-9c93-41caba12182a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:39.641126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cc5f0-7aa3-4de3-9c93-41caba12182a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:39.641408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cc5f0-7aa3-4de3-9c93-41caba12182a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:39.641734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cc5f0-7aa3-4de3-9c93-41caba12182a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:39.642986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cc5f0-7aa3-4de3-9c93-41caba12182a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:39.644500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cc5f0-7aa3-4de3-9c93-41caba12182a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:39.647202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 20/38] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:15:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:15:39.653026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da66bc-4bda-4e7d-9b61-1c334fe241f5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:39.704415 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7166340-5ed1-4baf-9c06-89f1a0f340a7 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:39.705005 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7166340-5ed1-4baf-9c06-89f1a0f340a7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:39.705904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7166340-5ed1-4baf-9c06-89f1a0f340a7 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:39.708877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 21/39] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:39.722215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97715-8dc5-4fc3-9ea8-8def958c9267 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:39.723628 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97715-8dc5-4fc3-9ea8-8def958c9267 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:39.724557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97715-8dc5-4fc3-9ea8-8def958c9267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:39.750350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-19a97715-8dc5-4fc3-9ea8-8def958c9267 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:39.752436 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-19a97715-8dc5-4fc3-9ea8-8def958c9267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:39.797180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da66bc-4bda-4e7d-9b61-1c334fe241f5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:39.831345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 22/40] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:15:39.832703 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b5369-2b14-41a4-983e-c539d21e034e None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:39.833048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b5369-2b14-41a4-983e-c539d21e034e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:39.833326 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b5369-2b14-41a4-983e-c539d21e034e None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:39.834737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 23/41] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:39.840351 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3a66d-2833-439a-b8f7-922691582288 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:39.840660 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3a66d-2833-439a-b8f7-922691582288 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:39.840992 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3a66d-2833-439a-b8f7-922691582288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:39.842873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8168217-2480-4bbe-a5bb-b1eaeca0832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:39.843559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8168217-2480-4bbe-a5bb-b1eaeca0832c None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:39.843846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8168217-2480-4bbe-a5bb-b1eaeca0832c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:39.844161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8168217-2480-4bbe-a5bb-b1eaeca0832c None admin] PATH_INFO: `/v3/domains/default/users/f615cddbb3154c329188ec780374eff5/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:39.846300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8168217-2480-4bbe-a5bb-b1eaeca0832c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f615cddbb3154c329188ec780374eff5, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:39.848693 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8168217-2480-4bbe-a5bb-b1eaeca0832c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:39.858686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:39.860237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-09b3a66d-2833-439a-b8f7-922691582288 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:39.861448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-09b3a66d-2833-439a-b8f7-922691582288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:39.876034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 19/42] 10.222.0.33 () {58 vars in 1361 bytes} [Sun Sep 11 05:15:39 2022] PUT /identity/v3/domains/default/users/f615cddbb3154c329188ec780374eff5/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:15:39.941384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 24/43] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:15:40.105226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8168217-2480-4bbe-a5bb-b1eaeca0832c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:40.207957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8168217-2480-4bbe-a5bb-b1eaeca0832c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:40.216674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3a66d-2833-439a-b8f7-922691582288 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:40.243245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48759c1-d6d0-4b13-b58c-5432c192f680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:40.244268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48759c1-d6d0-4b13-b58c-5432c192f680 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:40.244563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48759c1-d6d0-4b13-b58c-5432c192f680 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:40.244831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48759c1-d6d0-4b13-b58c-5432c192f680 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:40.245949 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48759c1-d6d0-4b13-b58c-5432c192f680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:40.247399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48759c1-d6d0-4b13-b58c-5432c192f680 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:40.249010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 20/44] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:15:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:40.254580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48759c1-d6d0-4b13-b58c-5432c192f680 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:40.327837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:40.371236 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:40.380771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48759c1-d6d0-4b13-b58c-5432c192f680 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:40.416366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d6a74-225d-4f5d-963f-0cfb10844cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:40.416989 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d6a74-225d-4f5d-963f-0cfb10844cac None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:40.417277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d6a74-225d-4f5d-963f-0cfb10844cac None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:40.417555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d6a74-225d-4f5d-963f-0cfb10844cac None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:40.418208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d6a74-225d-4f5d-963f-0cfb10844cac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:40.419668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d6a74-225d-4f5d-963f-0cfb10844cac None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:40.431686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3a66d-2833-439a-b8f7-922691582288 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:40.449301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 21/45] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:15:40 2022] POST /identity/v3/projects => generated 312 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:15:40.480496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee483e3-456a-4da6-b521-ce9f942a3b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:40.481150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee483e3-456a-4da6-b521-ce9f942a3b5d None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:40.481438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee483e3-456a-4da6-b521-ce9f942a3b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:40.481721 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee483e3-456a-4da6-b521-ce9f942a3b5d None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:40.482426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee483e3-456a-4da6-b521-ce9f942a3b5d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:40.483852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee483e3-456a-4da6-b521-ce9f942a3b5d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:40.504045 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 25/46] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 11 05:15:40 2022] POST /identity/v3/roles => generated 223 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:15:42.023820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f86a6-bde9-4601-a0b0-f669ae06c451 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:42.024393 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f86a6-bde9-4601-a0b0-f669ae06c451 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:42.024393 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f86a6-bde9-4601-a0b0-f669ae06c451 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:42.025744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 22/47] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:42.031115 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28aebb-c4e8-41da-95b5-401913035d06 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:42.031510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28aebb-c4e8-41da-95b5-401913035d06 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:42.031756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28aebb-c4e8-41da-95b5-401913035d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:42.049977 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-6a28aebb-c4e8-41da-95b5-401913035d06 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:42.051251 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6a28aebb-c4e8-41da-95b5-401913035d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:42.118786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 26/48] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:15:42.397703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f86a6-bde9-4601-a0b0-f669ae06c451 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:42.497167 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f86a6-bde9-4601-a0b0-f669ae06c451 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:42.514118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:42.532543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fb581-84f1-4041-80db-75eb58987d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:42.533210 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fb581-84f1-4041-80db-75eb58987d63 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:42.533494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fb581-84f1-4041-80db-75eb58987d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:42.533771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fb581-84f1-4041-80db-75eb58987d63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:42.546129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fb581-84f1-4041-80db-75eb58987d63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-9a5fb581-84f1-4041-80db-75eb58987d63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:15:42.551229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:42.552942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:42.553698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:42.553698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:42.553698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:42.553698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:42.553698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:15:42.553698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:15:42.553698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:15:42.553698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:15:42.553698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:15:42.553698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:15:42.553698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:15:42.553698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:15:42.553698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 23/49] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:15:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:15:42.559053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28aebb-c4e8-41da-95b5-401913035d06 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:42.592451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:42.653362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba948142-d292-4b65-b442-21aeac806ae1 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:42.653563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba948142-d292-4b65-b442-21aeac806ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:42.653836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba948142-d292-4b65-b442-21aeac806ae1 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:42.655192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 24/50] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:42.660138 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae2f54-c241-4323-ac96-eeaf63017d63 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:42.660421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae2f54-c241-4323-ac96-eeaf63017d63 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:42.660695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae2f54-c241-4323-ac96-eeaf63017d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:42.661605 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28aebb-c4e8-41da-95b5-401913035d06 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:42.679176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-dfae2f54-c241-4323-ac96-eeaf63017d63 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:42.680728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dfae2f54-c241-4323-ac96-eeaf63017d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:42.695071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479d174-ecad-4b2b-b190-aabeb186c778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:42.695685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479d174-ecad-4b2b-b190-aabeb186c778 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:42.695969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479d174-ecad-4b2b-b190-aabeb186c778 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:42.696279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479d174-ecad-4b2b-b190-aabeb186c778 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:42.696939 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479d174-ecad-4b2b-b190-aabeb186c778 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:42.697237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479d174-ecad-4b2b-b190-aabeb186c778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:42.698617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479d174-ecad-4b2b-b190-aabeb186c778 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:42.706742 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 27/51] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:15:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:42.713194 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479d174-ecad-4b2b-b190-aabeb186c778 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:42.747778 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 25/52] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:15:42.808897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:42.815054 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479d174-ecad-4b2b-b190-aabeb186c778 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:42.848231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0218e-248c-4c1b-a265-2b987f5fdb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:42.849428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0218e-248c-4c1b-a265-2b987f5fdb64 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:42.849716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0218e-248c-4c1b-a265-2b987f5fdb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:42.849985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0218e-248c-4c1b-a265-2b987f5fdb64 None admin] PATH_INFO: `/v3/users/f615cddbb3154c329188ec780374eff5` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:42.851170 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0218e-248c-4c1b-a265-2b987f5fdb64 None admin] RBAC: Authorizing `identity:get_user(user_id=f615cddbb3154c329188ec780374eff5)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:42.852649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0218e-248c-4c1b-a265-2b987f5fdb64 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:42.854211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 28/53] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:15:42 2022] GET /identity/v3/users/f615cddbb3154c329188ec780374eff5 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:15:42.859591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae2f54-c241-4323-ac96-eeaf63017d63 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:42.955579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae2f54-c241-4323-ac96-eeaf63017d63 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:43.018732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0218e-248c-4c1b-a265-2b987f5fdb64 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:43.067951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:43.094257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda689c4-5c78-411e-9396-707f99145bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:43.094898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda689c4-5c78-411e-9396-707f99145bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:43.095183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda689c4-5c78-411e-9396-707f99145bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:43.095462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda689c4-5c78-411e-9396-707f99145bc5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:43.096590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda689c4-5c78-411e-9396-707f99145bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:43.098043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda689c4-5c78-411e-9396-707f99145bc5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:43.099754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 26/54] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:15:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:15:43.105454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda689c4-5c78-411e-9396-707f99145bc5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:43.122645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0218e-248c-4c1b-a265-2b987f5fdb64 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:43.158179 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a2fc1-e2df-4633-a02f-07037e85271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:43.158811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a2fc1-e2df-4633-a02f-07037e85271f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:43.159091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a2fc1-e2df-4633-a02f-07037e85271f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:43.159364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a2fc1-e2df-4633-a02f-07037e85271f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:43.160492 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a2fc1-e2df-4633-a02f-07037e85271f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:43.162103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a2fc1-e2df-4633-a02f-07037e85271f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:43.163683 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 29/55] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:15:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:15:43.169225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a2fc1-e2df-4633-a02f-07037e85271f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:43.200218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:43.203507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda689c4-5c78-411e-9396-707f99145bc5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:43.236400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e0381-f0d8-4e90-93b9-267824de56ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:43.236947 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e0381-f0d8-4e90-93b9-267824de56ac None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:43.237247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e0381-f0d8-4e90-93b9-267824de56ac None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:43.237528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e0381-f0d8-4e90-93b9-267824de56ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:43.238258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e0381-f0d8-4e90-93b9-267824de56ac None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.domain.id=default, user.id=f615cddbb3154c329188ec780374eff5) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:43.238575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e0381-f0d8-4e90-93b9-267824de56ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:43.240126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e0381-f0d8-4e90-93b9-267824de56ac None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:43.248397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 27/56] 10.222.0.33 () {56 vars in 1322 bytes} [Sun Sep 11 05:15:43 2022] GET /identity/v3/role_assignments?user.id=f615cddbb3154c329188ec780374eff5&scope.domain.id=default&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:15:43.271640 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a2fc1-e2df-4633-a02f-07037e85271f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:43.308790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4ef8f-9422-49b5-9152-a0dd3c67ad5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:43.309432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4ef8f-9422-49b5-9152-a0dd3c67ad5b None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:43.309718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4ef8f-9422-49b5-9152-a0dd3c67ad5b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:43.309995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4ef8f-9422-49b5-9152-a0dd3c67ad5b None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:43.310680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4ef8f-9422-49b5-9152-a0dd3c67ad5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:43.312235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4ef8f-9422-49b5-9152-a0dd3c67ad5b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:43.343812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 30/57] 10.222.0.33 () {60 vars in 1140 bytes} [Sun Sep 11 05:15:43 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:15:45.494856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0d2e0-ae77-4462-941a-a25cc5b7d763 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:45.495369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0d2e0-ae77-4462-941a-a25cc5b7d763 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:45.495455 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0d2e0-ae77-4462-941a-a25cc5b7d763 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:45.496781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 28/58] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:45.502126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b128cb-8c3e-4069-a5e8-15e864ccd846 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:45.502542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b128cb-8c3e-4069-a5e8-15e864ccd846 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:45.502685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b128cb-8c3e-4069-a5e8-15e864ccd846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:45.521278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f3b128cb-8c3e-4069-a5e8-15e864ccd846 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:45.522486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f3b128cb-8c3e-4069-a5e8-15e864ccd846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:45.588184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 31/59] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:15:45.858845 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0d2e0-ae77-4462-941a-a25cc5b7d763 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:45.902037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:45.958981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0d2e0-ae77-4462-941a-a25cc5b7d763 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:45.992127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79751775-21f1-451c-a927-96faa79aae85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:45.992745 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79751775-21f1-451c-a927-96faa79aae85 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:45.993031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79751775-21f1-451c-a927-96faa79aae85 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:45.993310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79751775-21f1-451c-a927-96faa79aae85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:45.994428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79751775-21f1-451c-a927-96faa79aae85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:45.995883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79751775-21f1-451c-a927-96faa79aae85 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:45.997463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 29/60] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:15:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:46.002872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b128cb-8c3e-4069-a5e8-15e864ccd846 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:46.017646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:46.101492 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b128cb-8c3e-4069-a5e8-15e864ccd846 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:46.134254 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f70c62-5321-469b-94fd-29d04b5de2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:46.134848 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f70c62-5321-469b-94fd-29d04b5de2ac None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:46.135130 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f70c62-5321-469b-94fd-29d04b5de2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:46.135403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f70c62-5321-469b-94fd-29d04b5de2ac None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:46.136059 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f70c62-5321-469b-94fd-29d04b5de2ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:46.137501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f70c62-5321-469b-94fd-29d04b5de2ac None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:46.163451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 32/61] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:15:46 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:15:49.508154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b162b-a4f8-4e1d-9db7-c57275d9fbd8 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:49.509203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b162b-a4f8-4e1d-9db7-c57275d9fbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:49.509683 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b162b-a4f8-4e1d-9db7-c57275d9fbd8 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:49.511481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 30/62] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:49.517434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fe72a-5810-4a76-bae0-869135819e9e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:49.518308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fe72a-5810-4a76-bae0-869135819e9e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:49.518781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fe72a-5810-4a76-bae0-869135819e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:49.544425 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f88fe72a-5810-4a76-bae0-869135819e9e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:49.545970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f88fe72a-5810-4a76-bae0-869135819e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:49.637227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:49.642378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 33/63] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:15:49.720848 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2162465-8667-4949-ad7f-96a82086f88d None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:49.721481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2162465-8667-4949-ad7f-96a82086f88d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:49.721987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2162465-8667-4949-ad7f-96a82086f88d None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:49.724351 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 31/64] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:49.730426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02309d-ed73-4bde-b02b-b785c1008ee9 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:49.730710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02309d-ed73-4bde-b02b-b785c1008ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:49.731150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02309d-ed73-4bde-b02b-b785c1008ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:49.750909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ed02309d-ed73-4bde-b02b-b785c1008ee9 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:49.752386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ed02309d-ed73-4bde-b02b-b785c1008ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:49.835716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0057b4-f886-49b2-b2da-14a20375972c None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:49.836412 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0057b4-f886-49b2-b2da-14a20375972c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:49.836891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0057b4-f886-49b2-b2da-14a20375972c None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:49.839177 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 32/65] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:15:49.846741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea4fc1-5b47-4583-9ca3-1418063aa75c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:49.847169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 34/66] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:15:49.847568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea4fc1-5b47-4583-9ca3-1418063aa75c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:49.847851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea4fc1-5b47-4583-9ca3-1418063aa75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:49.877918 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f4ea4fc1-5b47-4583-9ca3-1418063aa75c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:49.880416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f4ea4fc1-5b47-4583-9ca3-1418063aa75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:49.964954 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90854598-ecb4-496c-8961-6b6639097bca None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:49.965579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90854598-ecb4-496c-8961-6b6639097bca None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:49.966052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90854598-ecb4-496c-8961-6b6639097bca None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:49.967677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 35/67] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:15:49.974579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 33/68] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:15:49.976191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec35a9-01f7-476d-a5af-e4d4f76856de None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:49.976682 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec35a9-01f7-476d-a5af-e4d4f76856de None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:49.977150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec35a9-01f7-476d-a5af-e4d4f76856de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:50.002130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea4fc1-5b47-4583-9ca3-1418063aa75c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:50.012278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-56ec35a9-01f7-476d-a5af-e4d4f76856de None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:50.013482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-56ec35a9-01f7-476d-a5af-e4d4f76856de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:50.053543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:50.113553 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea4fc1-5b47-4583-9ca3-1418063aa75c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:50.151522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c450b-1727-47fa-a3b4-4bfaf7599eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:50.152663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c450b-1727-47fa-a3b4-4bfaf7599eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:50.153098 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c450b-1727-47fa-a3b4-4bfaf7599eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:50.153521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c450b-1727-47fa-a3b4-4bfaf7599eb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:50.168774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c450b-1727-47fa-a3b4-4bfaf7599eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-288c450b-1727-47fa-a3b4-4bfaf7599eb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:15:50.178490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:50.179474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:50.180377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:50.180377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:50.180377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:50.180377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:50.180377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:15:50.180377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:15:50.180377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:15:50.180377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:15:50.180377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:15:50.180377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:15:50.180377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:15:50.180377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:15:50.183095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 34/69] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:15:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:15:50.191568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c450b-1727-47fa-a3b4-4bfaf7599eb7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:50.199235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 36/70] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:15:50.225353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a6d04-a8e6-4e92-bab1-55c66f12cbb7 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:50.225962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a6d04-a8e6-4e92-bab1-55c66f12cbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:50.226689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a6d04-a8e6-4e92-bab1-55c66f12cbb7 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:50.228352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 37/71] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:15:50.234656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a6d04-a8e6-4e92-bab1-55c66f12cbb7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:50.308345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:50.320743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:50.325603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c450b-1727-47fa-a3b4-4bfaf7599eb7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:50.360507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a6d04-a8e6-4e92-bab1-55c66f12cbb7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:50.364271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f52cf9-69ad-4571-bcdf-5cbfc9e88ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:50.364821 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f52cf9-69ad-4571-bcdf-5cbfc9e88ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:50.365104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f52cf9-69ad-4571-bcdf-5cbfc9e88ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:50.365380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f52cf9-69ad-4571-bcdf-5cbfc9e88ed6 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:50.366056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f52cf9-69ad-4571-bcdf-5cbfc9e88ed6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:50.366359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f52cf9-69ad-4571-bcdf-5cbfc9e88ed6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:50.367761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f52cf9-69ad-4571-bcdf-5cbfc9e88ed6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:50.385009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 35/72] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:15:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:15:50.385009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b451f-0346-4d16-882e-5a6208a856c9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:50.385009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b451f-0346-4d16-882e-5a6208a856c9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:50.385009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b451f-0346-4d16-882e-5a6208a856c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:50.400306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdb094-d467-4c9c-98bb-1b60bb16ed9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:50.400720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-dc5b451f-0346-4d16-882e-5a6208a856c9 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:50.401083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdb094-d467-4c9c-98bb-1b60bb16ed9f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:50.401461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdb094-d467-4c9c-98bb-1b60bb16ed9f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:50.401806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdb094-d467-4c9c-98bb-1b60bb16ed9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:50.401893 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dc5b451f-0346-4d16-882e-5a6208a856c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:50.413007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdb094-d467-4c9c-98bb-1b60bb16ed9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-0abdb094-d467-4c9c-98bb-1b60bb16ed9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:15:50.417908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:50.418737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:50.419494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:50.419494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:50.419494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:50.419494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:50.419494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:15:50.419494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:15:50.419494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:15:50.419494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:15:50.419494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:15:50.419494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:15:50.419494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:15:50.419494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:15:50.419494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 38/73] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:15:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:15:50.420171 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdb094-d467-4c9c-98bb-1b60bb16ed9f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:50.480067 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 36/74] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:15:50.480067 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b451f-0346-4d16-882e-5a6208a856c9 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:50.530212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdb094-d467-4c9c-98bb-1b60bb16ed9f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:50.551624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:50.571475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a0af6-9a78-40ee-adae-453c0e48b1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:50.572145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a0af6-9a78-40ee-adae-453c0e48b1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:50.572438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a0af6-9a78-40ee-adae-453c0e48b1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:50.572773 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a0af6-9a78-40ee-adae-453c0e48b1a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:50.576165 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:50.584105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a0af6-9a78-40ee-adae-453c0e48b1a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:15:50.587310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b451f-0346-4d16-882e-5a6208a856c9 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-e43a0af6-9a78-40ee-adae-453c0e48b1a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:15:50.588949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:50.589790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 39/75] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:15:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:15:50.592706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a0af6-9a78-40ee-adae-453c0e48b1a0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:50.631493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cddff-5552-487b-ba05-2ace3d37b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:50.632083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cddff-5552-487b-ba05-2ace3d37b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:50.632369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cddff-5552-487b-ba05-2ace3d37b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:50.632644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cddff-5552-487b-ba05-2ace3d37b7c3 None admin] PATH_INFO: `/v3/users/f001865a40c84901a03e7897036c3892` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:50.648785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cddff-5552-487b-ba05-2ace3d37b7c3 None admin] RBAC: Authorizing `identity:get_user(user_id=f001865a40c84901a03e7897036c3892)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:50.650368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cddff-5552-487b-ba05-2ace3d37b7c3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:50.656504 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 37/76] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:15:50 2022] GET /identity/v3/users/f001865a40c84901a03e7897036c3892 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:50.656504 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cddff-5552-487b-ba05-2ace3d37b7c3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:50.701453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a0af6-9a78-40ee-adae-453c0e48b1a0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:50.736530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f830079-53a3-45da-9910-f000c559cda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:50.737071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f830079-53a3-45da-9910-f000c559cda4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:50.737359 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f830079-53a3-45da-9910-f000c559cda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:50.737638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f830079-53a3-45da-9910-f000c559cda4 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:50.738319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f830079-53a3-45da-9910-f000c559cda4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:50.738628 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f830079-53a3-45da-9910-f000c559cda4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:50.740659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f830079-53a3-45da-9910-f000c559cda4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:50.748902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 40/77] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:15:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:50.753963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f830079-53a3-45da-9910-f000c559cda4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:50.766176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cddff-5552-487b-ba05-2ace3d37b7c3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:50.793828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:50.810521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cf8e5-d1be-4061-9fcb-0be84a7d9a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:50.811194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cf8e5-d1be-4061-9fcb-0be84a7d9a0f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:50.811475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cf8e5-d1be-4061-9fcb-0be84a7d9a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:50.811751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cf8e5-d1be-4061-9fcb-0be84a7d9a0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:50.821851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cf8e5-d1be-4061-9fcb-0be84a7d9a0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-ed5cf8e5-d1be-4061-9fcb-0be84a7d9a0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:15:50.827161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:50.828161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:50.828985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:50.828985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:50.828985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:50.828985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:50.828985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:15:50.828985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:15:50.828985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:15:50.828985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:15:50.828985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:15:50.828985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:15:50.828985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:15:50.828985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:15:50.828985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 38/78] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:15:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:15:50.829605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cf8e5-d1be-4061-9fcb-0be84a7d9a0f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:50.842036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:50.857090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f830079-53a3-45da-9910-f000c559cda4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:50.890390 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7779e-b087-4692-9ff1-97a51bb3fbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:50.891010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7779e-b087-4692-9ff1-97a51bb3fbde None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:50.891297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7779e-b087-4692-9ff1-97a51bb3fbde None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:50.891575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7779e-b087-4692-9ff1-97a51bb3fbde None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:50.892265 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7779e-b087-4692-9ff1-97a51bb3fbde None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:50.892571 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7779e-b087-4692-9ff1-97a51bb3fbde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:50.893991 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7779e-b087-4692-9ff1-97a51bb3fbde None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:50.902634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 41/79] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:15:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:50.903413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7779e-b087-4692-9ff1-97a51bb3fbde None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:50.933883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cf8e5-d1be-4061-9fcb-0be84a7d9a0f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:50.967628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ac8e1-4740-4426-9127-c5c237ebbecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:50.968271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ac8e1-4740-4426-9127-c5c237ebbecf None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:50.968558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ac8e1-4740-4426-9127-c5c237ebbecf None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:50.968838 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ac8e1-4740-4426-9127-c5c237ebbecf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:51.002070 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7779e-b087-4692-9ff1-97a51bb3fbde None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:51.004999 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:51.034782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ac8e1-4740-4426-9127-c5c237ebbecf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:51.036485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ac8e1-4740-4426-9127-c5c237ebbecf None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:51.038043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481d511-5a7c-4b09-bc4a-902e841908d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:51.038680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481d511-5a7c-4b09-bc4a-902e841908d1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:51.038962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481d511-5a7c-4b09-bc4a-902e841908d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:51.039243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481d511-5a7c-4b09-bc4a-902e841908d1 None admin] PATH_INFO: `/v3/users/f615cddbb3154c329188ec780374eff5` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:51.040379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481d511-5a7c-4b09-bc4a-902e841908d1 None admin] RBAC: Authorizing `identity:get_user(user_id=f615cddbb3154c329188ec780374eff5)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:51.041815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481d511-5a7c-4b09-bc4a-902e841908d1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:51.043367 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 42/80] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:15:50 2022] GET /identity/v3/users/f615cddbb3154c329188ec780374eff5 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:51.044249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481d511-5a7c-4b09-bc4a-902e841908d1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:51.046884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:51.134523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 39/81] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:15:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:15:51.135360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ac8e1-4740-4426-9127-c5c237ebbecf None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:51.149245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481d511-5a7c-4b09-bc4a-902e841908d1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:51.182730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffdfe8-0f95-4e2d-bd42-4195685cb0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:51.183362 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffdfe8-0f95-4e2d-bd42-4195685cb0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:51.183647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffdfe8-0f95-4e2d-bd42-4195685cb0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:51.183933 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffdfe8-0f95-4e2d-bd42-4195685cb0a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:51.185101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffdfe8-0f95-4e2d-bd42-4195685cb0a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:51.186553 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffdfe8-0f95-4e2d-bd42-4195685cb0a6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:51.188136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 43/82] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:15:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:51.188908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffdfe8-0f95-4e2d-bd42-4195685cb0a6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:51.230160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:51.234722 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ac8e1-4740-4426-9127-c5c237ebbecf None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:51.244535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:51.269116 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5842d-ac3b-487c-a20d-654e2b534c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:51.269721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5842d-ac3b-487c-a20d-654e2b534c20 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:51.270011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5842d-ac3b-487c-a20d-654e2b534c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:51.270510 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5842d-ac3b-487c-a20d-654e2b534c20 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:51.271233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5842d-ac3b-487c-a20d-654e2b534c20 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:51.271536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5842d-ac3b-487c-a20d-654e2b534c20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:51.272947 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5842d-ac3b-487c-a20d-654e2b534c20 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:51.280705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 40/83] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:15:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:51.286132 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5842d-ac3b-487c-a20d-654e2b534c20 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:51.289163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffdfe8-0f95-4e2d-bd42-4195685cb0a6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:51.323346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86048b23-fb5b-4d68-a407-c45a5f332383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:51.323967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86048b23-fb5b-4d68-a407-c45a5f332383 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:51.324286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86048b23-fb5b-4d68-a407-c45a5f332383 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:51.324568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86048b23-fb5b-4d68-a407-c45a5f332383 None admin] PATH_INFO: `/v3/users/f001865a40c84901a03e7897036c3892` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:51.325734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86048b23-fb5b-4d68-a407-c45a5f332383 None admin] RBAC: Authorizing `identity:get_user(user_id=f001865a40c84901a03e7897036c3892)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:51.327189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86048b23-fb5b-4d68-a407-c45a5f332383 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:51.328968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 44/84] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:15:51 2022] GET /identity/v3/users/f001865a40c84901a03e7897036c3892 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:51.329566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86048b23-fb5b-4d68-a407-c45a5f332383 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:51.387291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5842d-ac3b-487c-a20d-654e2b534c20 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:51.419969 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe21bd-2715-479d-bf4c-2cd4bf857a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:51.420533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe21bd-2715-479d-bf4c-2cd4bf857a9a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:51.420815 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe21bd-2715-479d-bf4c-2cd4bf857a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:51.421087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe21bd-2715-479d-bf4c-2cd4bf857a9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:51.430405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86048b23-fb5b-4d68-a407-c45a5f332383 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:51.439720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:51.461236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:51.466971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4364f7e-af39-4191-9e57-29200d6e97ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:51.467606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364f7e-af39-4191-9e57-29200d6e97ed None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:51.467890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364f7e-af39-4191-9e57-29200d6e97ed None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:51.468207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364f7e-af39-4191-9e57-29200d6e97ed None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:51.469325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4364f7e-af39-4191-9e57-29200d6e97ed None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:51.470798 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4364f7e-af39-4191-9e57-29200d6e97ed None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:51.472493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 45/85] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:15:51 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:51.473382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4364f7e-af39-4191-9e57-29200d6e97ed None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:51.490195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe21bd-2715-479d-bf4c-2cd4bf857a9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:51.491692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe21bd-2715-479d-bf4c-2cd4bf857a9a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:51.573685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4364f7e-af39-4191-9e57-29200d6e97ed None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:51.592025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 41/86] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:15:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:15:51.593453 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe21bd-2715-479d-bf4c-2cd4bf857a9a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:51.607749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376efe3-e9e8-45ae-af1e-ed30b49251e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:51.608392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376efe3-e9e8-45ae-af1e-ed30b49251e6 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:51.608679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376efe3-e9e8-45ae-af1e-ed30b49251e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:51.608978 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376efe3-e9e8-45ae-af1e-ed30b49251e6 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:51.609617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376efe3-e9e8-45ae-af1e-ed30b49251e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:51.611082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376efe3-e9e8-45ae-af1e-ed30b49251e6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:51.634213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 46/87] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:15:51 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:15:51.635023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376efe3-e9e8-45ae-af1e-ed30b49251e6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:51.668646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:51.671537 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:51.705210 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe21bd-2715-479d-bf4c-2cd4bf857a9a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:51.742941 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45d65e-2439-4d8e-a64b-05cc44764983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:51.743598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45d65e-2439-4d8e-a64b-05cc44764983 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:51.743887 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45d65e-2439-4d8e-a64b-05cc44764983 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:51.744200 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45d65e-2439-4d8e-a64b-05cc44764983 None admin] PATH_INFO: `/v3/users/f001865a40c84901a03e7897036c3892` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:51.745414 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45d65e-2439-4d8e-a64b-05cc44764983 None admin] RBAC: Authorizing `identity:get_user(user_id=f001865a40c84901a03e7897036c3892)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:51.746901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45d65e-2439-4d8e-a64b-05cc44764983 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:51.753170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 42/88] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:15:51 2022] GET /identity/v3/users/f001865a40c84901a03e7897036c3892 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:51.753170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45d65e-2439-4d8e-a64b-05cc44764983 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:51.753701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376efe3-e9e8-45ae-af1e-ed30b49251e6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:51.797367 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60b1dd-3395-43e7-8c8f-9c237b3fdb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:51.798017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60b1dd-3395-43e7-8c8f-9c237b3fdb32 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:51.798321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60b1dd-3395-43e7-8c8f-9c237b3fdb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:51.798586 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60b1dd-3395-43e7-8c8f-9c237b3fdb32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:51.870564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45d65e-2439-4d8e-a64b-05cc44764983 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:51.886907 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60b1dd-3395-43e7-8c8f-9c237b3fdb32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:51.888958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60b1dd-3395-43e7-8c8f-9c237b3fdb32 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:51.914056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af76d4-5867-445e-94c6-2e4bbcaeadbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:51.914746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af76d4-5867-445e-94c6-2e4bbcaeadbc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:51.915094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af76d4-5867-445e-94c6-2e4bbcaeadbc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:51.915379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af76d4-5867-445e-94c6-2e4bbcaeadbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:51.916161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af76d4-5867-445e-94c6-2e4bbcaeadbc None admin] RBAC: Adding query filter params (role.id=d9f6babaca2d494c9a04f0aca1271af8, scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3, user.id=f001865a40c84901a03e7897036c3892) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:51.916476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af76d4-5867-445e-94c6-2e4bbcaeadbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:51.918131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af76d4-5867-445e-94c6-2e4bbcaeadbc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:51.925179 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 43/89] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:15:51 2022] GET /identity/v3/role_assignments?user.id=f001865a40c84901a03e7897036c3892&scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3&role.id=d9f6babaca2d494c9a04f0aca1271af8 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:51.925795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af76d4-5867-445e-94c6-2e4bbcaeadbc None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:51.926473 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:51.930599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:52.021561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 47/90] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:15:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:15:52.022464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60b1dd-3395-43e7-8c8f-9c237b3fdb32 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:52.072057 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af76d4-5867-445e-94c6-2e4bbcaeadbc None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:52.139181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692326c7-c188-4fcf-b3f4-f0af7d4eae49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:52.139832 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692326c7-c188-4fcf-b3f4-f0af7d4eae49 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:52.140145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692326c7-c188-4fcf-b3f4-f0af7d4eae49 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:52.140421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692326c7-c188-4fcf-b3f4-f0af7d4eae49 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:52.141565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692326c7-c188-4fcf-b3f4-f0af7d4eae49 None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:52.143001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692326c7-c188-4fcf-b3f4-f0af7d4eae49 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:52.144673 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 44/91] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:15:51 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:52.145830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692326c7-c188-4fcf-b3f4-f0af7d4eae49 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:52.178462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60b1dd-3395-43e7-8c8f-9c237b3fdb32 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:52.213182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:52.217032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0150e-c13e-48c1-b356-c5ebf06e68af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:52.217691 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0150e-c13e-48c1-b356-c5ebf06e68af None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:52.218081 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0150e-c13e-48c1-b356-c5ebf06e68af None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:52.218369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0150e-c13e-48c1-b356-c5ebf06e68af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:52.221070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:52.254745 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692326c7-c188-4fcf-b3f4-f0af7d4eae49 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:52.292414 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e739f-9042-45f4-8883-7144fadba494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:52.293055 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e739f-9042-45f4-8883-7144fadba494 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:52.293333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e739f-9042-45f4-8883-7144fadba494 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:52.293606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e739f-9042-45f4-8883-7144fadba494 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:52.295411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e739f-9042-45f4-8883-7144fadba494 None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:52.296883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e739f-9042-45f4-8883-7144fadba494 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:52.300161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 45/92] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:15:52 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:52.300659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e739f-9042-45f4-8883-7144fadba494 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:52.324031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0150e-c13e-48c1-b356-c5ebf06e68af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:52.325637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0150e-c13e-48c1-b356-c5ebf06e68af None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:52.497317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e739f-9042-45f4-8883-7144fadba494 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:52.500977 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 48/93] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:15:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:15:52.502186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0150e-c13e-48c1-b356-c5ebf06e68af None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:52.559572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:52.565358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380061e7-4430-4998-9345-8ccdd5b132b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:52.566065 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380061e7-4430-4998-9345-8ccdd5b132b4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:52.572018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380061e7-4430-4998-9345-8ccdd5b132b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:52.572018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380061e7-4430-4998-9345-8ccdd5b132b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:52.572018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380061e7-4430-4998-9345-8ccdd5b132b4 None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3, user.id=f615cddbb3154c329188ec780374eff5) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:52.572018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380061e7-4430-4998-9345-8ccdd5b132b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:52.572018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380061e7-4430-4998-9345-8ccdd5b132b4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:52.580021 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 46/94] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:15:52 2022] GET /identity/v3/role_assignments?user.id=f615cddbb3154c329188ec780374eff5&scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:52.580021 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380061e7-4430-4998-9345-8ccdd5b132b4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:52.596472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:52.682909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0150e-c13e-48c1-b356-c5ebf06e68af None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:52.733581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e035863-c7bf-45df-86e8-83aa2b1bd4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:52.734232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e035863-c7bf-45df-86e8-83aa2b1bd4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:52.734519 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e035863-c7bf-45df-86e8-83aa2b1bd4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:52.734793 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e035863-c7bf-45df-86e8-83aa2b1bd4e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:52.735550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e035863-c7bf-45df-86e8-83aa2b1bd4e4 None admin] RBAC: Adding query filter params (role.id=6c6c7fa19ee342da915c90895e707620, scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3, user.id=f001865a40c84901a03e7897036c3892) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:52.735862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e035863-c7bf-45df-86e8-83aa2b1bd4e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:52.737380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e035863-c7bf-45df-86e8-83aa2b1bd4e4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:52.764884 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 49/95] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:15:52 2022] GET /identity/v3/role_assignments?user.id=f001865a40c84901a03e7897036c3892&scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3&role.id=6c6c7fa19ee342da915c90895e707620 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:52.835144 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380061e7-4430-4998-9345-8ccdd5b132b4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:52.893049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796073fd-2327-4669-aa71-5a63278ac8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:52.893707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796073fd-2327-4669-aa71-5a63278ac8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:52.893995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796073fd-2327-4669-aa71-5a63278ac8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:52.894271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796073fd-2327-4669-aa71-5a63278ac8d2 None admin] PATH_INFO: `/v3/projects/fe38e48af40b4c28a6808311efb73ac5` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:52.895512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796073fd-2327-4669-aa71-5a63278ac8d2 None admin] RBAC: Authorizing `identity:get_project(project_id=fe38e48af40b4c28a6808311efb73ac5)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:52.897033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796073fd-2327-4669-aa71-5a63278ac8d2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:52.899068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 47/96] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:15:52 2022] GET /identity/v3/projects/fe38e48af40b4c28a6808311efb73ac5 => generated 312 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:52.904983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e035863-c7bf-45df-86e8-83aa2b1bd4e4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:53.057725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:53.066042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e035863-c7bf-45df-86e8-83aa2b1bd4e4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:53.117514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b72f00-7073-403c-96d5-854b3e96fb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:53.118160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b72f00-7073-403c-96d5-854b3e96fb00 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:53.118446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b72f00-7073-403c-96d5-854b3e96fb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:53.118718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b72f00-7073-403c-96d5-854b3e96fb00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:53.119538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b72f00-7073-403c-96d5-854b3e96fb00 None admin] RBAC: Adding query filter params (role.id=d9f6babaca2d494c9a04f0aca1271af8, scope.project.id=fe38e48af40b4c28a6808311efb73ac5, user.id=f001865a40c84901a03e7897036c3892) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:53.119852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b72f00-7073-403c-96d5-854b3e96fb00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:53.121429 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b72f00-7073-403c-96d5-854b3e96fb00 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:53.132156 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 50/97] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:15:52 2022] GET /identity/v3/role_assignments?user.id=f001865a40c84901a03e7897036c3892&scope.project.id=fe38e48af40b4c28a6808311efb73ac5&role.id=d9f6babaca2d494c9a04f0aca1271af8 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:54.646267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ac2e5-7863-4d72-b10e-4e4a7579bdd6 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:54.646807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ac2e5-7863-4d72-b10e-4e4a7579bdd6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:54.646807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ac2e5-7863-4d72-b10e-4e4a7579bdd6 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:54.648305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 48/98] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:54.662769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f075dc6-0c0e-496d-8909-2ed32726b14a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:54.663223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f075dc6-0c0e-496d-8909-2ed32726b14a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:54.663322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f075dc6-0c0e-496d-8909-2ed32726b14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:54.685673 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b5cb4-be76-4722-91a6-bebd10848127 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:54.685955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b5cb4-be76-4722-91a6-bebd10848127 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:54.686239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b5cb4-be76-4722-91a6-bebd10848127 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:54.687580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 49/99] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:15:54.689681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9f075dc6-0c0e-496d-8909-2ed32726b14a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:54.690985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9f075dc6-0c0e-496d-8909-2ed32726b14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:54.697224 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e168715-ca7b-4c6d-9ee7-eecdeb952342 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:54.697828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e168715-ca7b-4c6d-9ee7-eecdeb952342 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:54.698115 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e168715-ca7b-4c6d-9ee7-eecdeb952342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:54.730245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8e168715-ca7b-4c6d-9ee7-eecdeb952342 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:54.731452 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8e168715-ca7b-4c6d-9ee7-eecdeb952342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:54.794355 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 51/100] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:15:54.868955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:54.875261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 50/101] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:15:55.163044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f075dc6-0c0e-496d-8909-2ed32726b14a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:55.290870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:55.302446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f075dc6-0c0e-496d-8909-2ed32726b14a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:55.349244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb02dd0-4ddf-4ced-bfa3-6c96bd27b7f3 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:55.350062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb02dd0-4ddf-4ced-bfa3-6c96bd27b7f3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:55.350546 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb02dd0-4ddf-4ced-bfa3-6c96bd27b7f3 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:55.351185 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a807e-8915-4aec-9579-8f8f7bb0456a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:55.351833 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a807e-8915-4aec-9579-8f8f7bb0456a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:55.352141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a807e-8915-4aec-9579-8f8f7bb0456a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:55.352416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a807e-8915-4aec-9579-8f8f7bb0456a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:55.353123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 51/102] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:15:55.358419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6db714-31cd-4e9e-a6f9-ddef2f4a1a0e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:55.358951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6db714-31cd-4e9e-a6f9-ddef2f4a1a0e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:55.359428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6db714-31cd-4e9e-a6f9-ddef2f4a1a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:55.363351 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a807e-8915-4aec-9579-8f8f7bb0456a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-f09a807e-8915-4aec-9579-8f8f7bb0456a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:15:55.368754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:55.369602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:55.370368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:55.370368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:55.370368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:55.370368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:55.370368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:15:55.370368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:15:55.370368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:15:55.370368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:15:55.370368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:15:55.370368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:15:55.370368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:15:55.370368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:15:55.370368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 52/103] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:15:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:15:55.376113 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a807e-8915-4aec-9579-8f8f7bb0456a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:55.396868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0c6db714-31cd-4e9e-a6f9-ddef2f4a1a0e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:55.399986 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0c6db714-31cd-4e9e-a6f9-ddef2f4a1a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:55.529996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 52/104] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:15:55.530969 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6db714-31cd-4e9e-a6f9-ddef2f4a1a0e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:55.571690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a807e-8915-4aec-9579-8f8f7bb0456a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:55.643462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4d659-07ad-4463-a686-3f6598c58299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:55.644112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4d659-07ad-4463-a686-3f6598c58299 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:55.644395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4d659-07ad-4463-a686-3f6598c58299 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:55.644666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4d659-07ad-4463-a686-3f6598c58299 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:55.646238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4d659-07ad-4463-a686-3f6598c58299 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:55.646664 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4d659-07ad-4463-a686-3f6598c58299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:55.648091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4d659-07ad-4463-a686-3f6598c58299 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:55.667176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 53/105] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:15:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:55.679690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4d659-07ad-4463-a686-3f6598c58299 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:55.712718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:55.752221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6db714-31cd-4e9e-a6f9-ddef2f4a1a0e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:55.827693 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:55.838551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34897cd-603e-47b5-98f1-0af0d1184797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:55.839191 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34897cd-603e-47b5-98f1-0af0d1184797 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:55.839470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34897cd-603e-47b5-98f1-0af0d1184797 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:55.839738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34897cd-603e-47b5-98f1-0af0d1184797 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:55.861333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34897cd-603e-47b5-98f1-0af0d1184797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d34897cd-603e-47b5-98f1-0af0d1184797 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:15:55.866644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:55.867483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:55.868282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:55.868282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:55.868282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:55.868282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:55.868282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:15:55.868282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:15:55.868282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:15:55.868282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:15:55.868282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:15:55.868282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:15:55.868282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:15:55.868282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:15:55.868282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 53/106] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:15:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:15:55.868934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2e2e6-5540-47f2-b310-4504b58d6171 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:55.869738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2e2e6-5540-47f2-b310-4504b58d6171 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:55.870874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2e2e6-5540-47f2-b310-4504b58d6171 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:55.878691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 54/107] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:55.885142 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2e2e6-5540-47f2-b310-4504b58d6171 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:55.902790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4d659-07ad-4463-a686-3f6598c58299 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:55.940730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974bc1e-1c92-41c3-8e21-5c55510f8f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:55.941526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974bc1e-1c92-41c3-8e21-5c55510f8f67 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:55.942444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974bc1e-1c92-41c3-8e21-5c55510f8f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:55.942724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974bc1e-1c92-41c3-8e21-5c55510f8f67 None admin] PATH_INFO: `/v3/users/f152a86ae2354080a31a2daaeec2ef96` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:55.957846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974bc1e-1c92-41c3-8e21-5c55510f8f67 None admin] RBAC: Authorizing `identity:get_user(user_id=f152a86ae2354080a31a2daaeec2ef96)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:55.959359 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974bc1e-1c92-41c3-8e21-5c55510f8f67 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:55.961583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 54/108] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:15:55 2022] GET /identity/v3/users/f152a86ae2354080a31a2daaeec2ef96 => generated 289 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:55.962369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974bc1e-1c92-41c3-8e21-5c55510f8f67 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:56.000450 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2e2e6-5540-47f2-b310-4504b58d6171 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:56.039706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:56.042705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10774b40-85e9-48e7-80f4-22532bc69205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:56.043330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10774b40-85e9-48e7-80f4-22532bc69205 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:56.043618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10774b40-85e9-48e7-80f4-22532bc69205 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:56.043885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10774b40-85e9-48e7-80f4-22532bc69205 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:56.044616 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10774b40-85e9-48e7-80f4-22532bc69205 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:56.044919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10774b40-85e9-48e7-80f4-22532bc69205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:56.046953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10774b40-85e9-48e7-80f4-22532bc69205 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:56.058208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 55/109] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:15:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:56.059983 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e3e4d-7471-4316-8181-d369cdafdbd7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:56.060293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e3e4d-7471-4316-8181-d369cdafdbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:56.060569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e3e4d-7471-4316-8181-d369cdafdbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:56.076930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974bc1e-1c92-41c3-8e21-5c55510f8f67 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:56.089626 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fe9e3e4d-7471-4316-8181-d369cdafdbd7 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:56.090433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fe9e3e4d-7471-4316-8181-d369cdafdbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:56.121758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c57ee-79bd-424e-a268-8a09c77bfa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:56.122409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c57ee-79bd-424e-a268-8a09c77bfa21 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:56.122695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c57ee-79bd-424e-a268-8a09c77bfa21 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:56.122973 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c57ee-79bd-424e-a268-8a09c77bfa21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:56.124347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c57ee-79bd-424e-a268-8a09c77bfa21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:56.125907 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c57ee-79bd-424e-a268-8a09c77bfa21 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:56.127587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 55/110] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:15:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:56.129515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c57ee-79bd-424e-a268-8a09c77bfa21 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:56.152053 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:56.184036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 56/111] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:15:56.184036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e3e4d-7471-4316-8181-d369cdafdbd7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:56.264648 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c57ee-79bd-424e-a268-8a09c77bfa21 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:56.306202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e3e4d-7471-4316-8181-d369cdafdbd7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:56.308308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265c1fd-c55c-45f9-86fa-6d8544b47ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:56.309142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265c1fd-c55c-45f9-86fa-6d8544b47ddc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:56.309636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265c1fd-c55c-45f9-86fa-6d8544b47ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:56.310162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265c1fd-c55c-45f9-86fa-6d8544b47ddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:56.319157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:56.344398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba676b95-ad13-4afd-90a4-04a0dbcc713a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:56.345047 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba676b95-ad13-4afd-90a4-04a0dbcc713a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:56.345331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba676b95-ad13-4afd-90a4-04a0dbcc713a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:56.345608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba676b95-ad13-4afd-90a4-04a0dbcc713a None admin] PATH_INFO: `/v3/users/f001865a40c84901a03e7897036c3892` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:56.346752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba676b95-ad13-4afd-90a4-04a0dbcc713a None admin] RBAC: Authorizing `identity:get_user(user_id=f001865a40c84901a03e7897036c3892)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:56.348446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba676b95-ad13-4afd-90a4-04a0dbcc713a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:56.350450 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 57/112] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:15:56 2022] GET /identity/v3/users/f001865a40c84901a03e7897036c3892 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:56.351106 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba676b95-ad13-4afd-90a4-04a0dbcc713a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:56.387928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265c1fd-c55c-45f9-86fa-6d8544b47ddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:56.389561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265c1fd-c55c-45f9-86fa-6d8544b47ddc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:56.408705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:56.460181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba676b95-ad13-4afd-90a4-04a0dbcc713a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:56.494976 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c8a0f-5c73-4f33-bdb9-91dfa3c1cd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:56.495579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c8a0f-5c73-4f33-bdb9-91dfa3c1cd14 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:56.495856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c8a0f-5c73-4f33-bdb9-91dfa3c1cd14 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:56.496157 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c8a0f-5c73-4f33-bdb9-91dfa3c1cd14 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:56.496800 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c8a0f-5c73-4f33-bdb9-91dfa3c1cd14 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:56.498232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c8a0f-5c73-4f33-bdb9-91dfa3c1cd14 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:56.498576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 56/113] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:15:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:15:56.503137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd4d6d-6231-46ea-9016-ded282c47673 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:56.503137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd4d6d-6231-46ea-9016-ded282c47673 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:56.503137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd4d6d-6231-46ea-9016-ded282c47673 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:56.503137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 57/114] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:56.503137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03722a-5e67-489b-9ff0-12bcac26bea8 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:56.503137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03722a-5e67-489b-9ff0-12bcac26bea8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:56.503831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03722a-5e67-489b-9ff0-12bcac26bea8 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:56.504633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 58/115] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:56.505340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03722a-5e67-489b-9ff0-12bcac26bea8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:56.526396 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 58/116] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:15:56 2022] POST /identity/v3/projects => generated 302 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:15:56.527287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c8a0f-5c73-4f33-bdb9-91dfa3c1cd14 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:56.556022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:56.629112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03722a-5e67-489b-9ff0-12bcac26bea8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:56.646978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c8a0f-5c73-4f33-bdb9-91dfa3c1cd14 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:56.648829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:56.669457 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422310b8-c141-4c8a-9437-66c12484460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:56.670109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422310b8-c141-4c8a-9437-66c12484460b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:56.670348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422310b8-c141-4c8a-9437-66c12484460b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:56.670623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422310b8-c141-4c8a-9437-66c12484460b None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:56.671767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422310b8-c141-4c8a-9437-66c12484460b None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:56.673300 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422310b8-c141-4c8a-9437-66c12484460b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:56.675265 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 59/117] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:15:56 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:56.675848 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422310b8-c141-4c8a-9437-66c12484460b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:56.695555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd1d6b-17b7-4adc-b108-69051c373b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:56.696251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd1d6b-17b7-4adc-b108-69051c373b47 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:56.696545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd1d6b-17b7-4adc-b108-69051c373b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:56.696996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd1d6b-17b7-4adc-b108-69051c373b47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:56.710726 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd1d6b-17b7-4adc-b108-69051c373b47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-f5cd1d6b-17b7-4adc-b108-69051c373b47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:15:56.715670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:56.716554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 59/118] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:15:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:15:56.718379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19996079-500e-4a6e-9a0c-088ceb20763b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:56.719072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19996079-500e-4a6e-9a0c-088ceb20763b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:56.719072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19996079-500e-4a6e-9a0c-088ceb20763b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:56.737144 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-19996079-500e-4a6e-9a0c-088ceb20763b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:56.738785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-19996079-500e-4a6e-9a0c-088ceb20763b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:56.794729 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422310b8-c141-4c8a-9437-66c12484460b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:56.804353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:56.815580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 60/119] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:15:56.822533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827158d-9ac3-496a-b45c-f5c07f3c9432 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:56.822833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827158d-9ac3-496a-b45c-f5c07f3c9432 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:56.823247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827158d-9ac3-496a-b45c-f5c07f3c9432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:56.845626 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d827158d-9ac3-496a-b45c-f5c07f3c9432 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:56.845841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d19c9c-89d4-4249-b9df-ab0a917f0af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:56.846615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d19c9c-89d4-4249-b9df-ab0a917f0af7 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:56.846825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d827158d-9ac3-496a-b45c-f5c07f3c9432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:56.847048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d19c9c-89d4-4249-b9df-ab0a917f0af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:56.847430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d19c9c-89d4-4249-b9df-ab0a917f0af7 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:56.849103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d19c9c-89d4-4249-b9df-ab0a917f0af7 None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:56.850690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d19c9c-89d4-4249-b9df-ab0a917f0af7 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:56.852512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 60/120] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:15:56 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:56.853332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d19c9c-89d4-4249-b9df-ab0a917f0af7 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:56.936111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 61/121] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:15:56.937722 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827158d-9ac3-496a-b45c-f5c07f3c9432 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:56.965728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:56.969023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d19c9c-89d4-4249-b9df-ab0a917f0af7 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:57.032828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5351f-b243-424d-9eb8-bcf87ab1e448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:57.033500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5351f-b243-424d-9eb8-bcf87ab1e448 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:57.033782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5351f-b243-424d-9eb8-bcf87ab1e448 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:57.034060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5351f-b243-424d-9eb8-bcf87ab1e448 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3/users/f001865a40c84901a03e7897036c3892/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:57.037795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5351f-b243-424d-9eb8-bcf87ab1e448 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3, user_id=f001865a40c84901a03e7897036c3892, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:57.039257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5351f-b243-424d-9eb8-bcf87ab1e448 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:57.060097 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 61/122] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:15:56 2022] PUT /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3/users/f001865a40c84901a03e7897036c3892/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:15:57.060806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5351f-b243-424d-9eb8-bcf87ab1e448 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:57.103246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:57.131388 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827158d-9ac3-496a-b45c-f5c07f3c9432 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:57.181533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5351f-b243-424d-9eb8-bcf87ab1e448 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:57.219192 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70868b83-77ea-434e-bcdf-05fd5e284f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:57.219814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70868b83-77ea-434e-bcdf-05fd5e284f51 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:57.220141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70868b83-77ea-434e-bcdf-05fd5e284f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:57.220415 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70868b83-77ea-434e-bcdf-05fd5e284f51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:57.221160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70868b83-77ea-434e-bcdf-05fd5e284f51 None admin] RBAC: Adding query filter params (role.id=8cf91ce8e7dd42aaa2708185cb13a6db, scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3, user.id=f152a86ae2354080a31a2daaeec2ef96) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:57.221470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70868b83-77ea-434e-bcdf-05fd5e284f51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:57.223728 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70868b83-77ea-434e-bcdf-05fd5e284f51 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:57.230579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 62/123] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:15:57 2022] GET /identity/v3/role_assignments?user.id=f152a86ae2354080a31a2daaeec2ef96&scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3&role.id=8cf91ce8e7dd42aaa2708185cb13a6db => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:57.234507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cbef7-0f04-44a7-8120-09bcd8ac5a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:57.235155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cbef7-0f04-44a7-8120-09bcd8ac5a45 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:57.235437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cbef7-0f04-44a7-8120-09bcd8ac5a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:57.235710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cbef7-0f04-44a7-8120-09bcd8ac5a45 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:57.236425 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0cbef7-0f04-44a7-8120-09bcd8ac5a45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:57.236733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0cbef7-0f04-44a7-8120-09bcd8ac5a45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:57.238268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0cbef7-0f04-44a7-8120-09bcd8ac5a45 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:57.249342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70868b83-77ea-434e-bcdf-05fd5e284f51 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:57.261300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 62/124] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:15:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:57.273315 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cbef7-0f04-44a7-8120-09bcd8ac5a45 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:57.410463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:57.415294 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70868b83-77ea-434e-bcdf-05fd5e284f51 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:57.440969 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:57.458849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f6f57-7fe4-4563-9daa-db8b04de010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:57.459513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f6f57-7fe4-4563-9daa-db8b04de010e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:57.459800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f6f57-7fe4-4563-9daa-db8b04de010e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:57.461784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f6f57-7fe4-4563-9daa-db8b04de010e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:57.462740 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cbef7-0f04-44a7-8120-09bcd8ac5a45 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:57.473768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f6f57-7fe4-4563-9daa-db8b04de010e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-9f3f6f57-7fe4-4563-9daa-db8b04de010e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:15:57.479411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:57.480381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:57.481145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:57.481145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:57.481145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:57.481145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:57.481145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:15:57.481145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:15:57.481145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:15:57.481145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:15:57.481145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:15:57.481145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:15:57.481145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:15:57.481145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:15:57.481145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 63/125] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:15:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:15:57.481778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f6f57-7fe4-4563-9daa-db8b04de010e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:57.505946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f4c5c-8ede-4cbf-a1eb-f87abe5f1389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:57.506592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f4c5c-8ede-4cbf-a1eb-f87abe5f1389 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:57.506881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f4c5c-8ede-4cbf-a1eb-f87abe5f1389 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:57.507734 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f4c5c-8ede-4cbf-a1eb-f87abe5f1389 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:57.520205 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f4c5c-8ede-4cbf-a1eb-f87abe5f1389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-2b3f4c5c-8ede-4cbf-a1eb-f87abe5f1389 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:15:57.527711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:15:57.528635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:15:57.529401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:15:57.529401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:15:57.529401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:15:57.529401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:15:57.529401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:15:57.529401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:15:57.529401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:15:57.529401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:15:57.529401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:15:57.529401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:15:57.529401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:15:57.529401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:15:57.529401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 63/126] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:15:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:15:57.536022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f4c5c-8ede-4cbf-a1eb-f87abe5f1389 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:57.640887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f6f57-7fe4-4563-9daa-db8b04de010e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:57.652788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f4c5c-8ede-4cbf-a1eb-f87abe5f1389 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:57.696782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b6eba-d8c6-4175-ab38-9fd4c01cc63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:57.697475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b6eba-d8c6-4175-ab38-9fd4c01cc63c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:57.697909 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b6eba-d8c6-4175-ab38-9fd4c01cc63c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:57.698326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b6eba-d8c6-4175-ab38-9fd4c01cc63c None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:57.699041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b6eba-d8c6-4175-ab38-9fd4c01cc63c None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:57.699351 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b6eba-d8c6-4175-ab38-9fd4c01cc63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:57.700891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b6eba-d8c6-4175-ab38-9fd4c01cc63c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:57.706035 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399ae54-a697-46bb-9da9-7e651cb79c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:57.706689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ae54-a697-46bb-9da9-7e651cb79c0d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:57.706977 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ae54-a697-46bb-9da9-7e651cb79c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:57.707292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ae54-a697-46bb-9da9-7e651cb79c0d None admin] PATH_INFO: `/v3/users/f615cddbb3154c329188ec780374eff5` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:57.708530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399ae54-a697-46bb-9da9-7e651cb79c0d None admin] RBAC: Authorizing `identity:get_user(user_id=f615cddbb3154c329188ec780374eff5)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:57.713343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399ae54-a697-46bb-9da9-7e651cb79c0d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:57.714971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 64/127] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:15:57 2022] GET /identity/v3/users/f615cddbb3154c329188ec780374eff5 => generated 247 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:57.717873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399ae54-a697-46bb-9da9-7e651cb79c0d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:57.720133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 64/128] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:15:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:57.720849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b6eba-d8c6-4175-ab38-9fd4c01cc63c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:57.735494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:57.786999 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:57.841715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b6eba-d8c6-4175-ab38-9fd4c01cc63c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:57.907306 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c18cef-e4d9-4bfb-b588-0cf933a7b5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:57.911347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c18cef-e4d9-4bfb-b588-0cf933a7b5bf None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:57.911803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c18cef-e4d9-4bfb-b588-0cf933a7b5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:57.912256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c18cef-e4d9-4bfb-b588-0cf933a7b5bf None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:57.913668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c18cef-e4d9-4bfb-b588-0cf933a7b5bf None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:57.915289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c18cef-e4d9-4bfb-b588-0cf933a7b5bf None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:57.920607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 65/129] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:15:57 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:57.921807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c18cef-e4d9-4bfb-b588-0cf933a7b5bf None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:57.945623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399ae54-a697-46bb-9da9-7e651cb79c0d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:58.020680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:58.025983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524711e-6ec9-4d24-b013-e2cd34303e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:58.026995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524711e-6ec9-4d24-b013-e2cd34303e26 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:58.027431 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524711e-6ec9-4d24-b013-e2cd34303e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:58.027927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524711e-6ec9-4d24-b013-e2cd34303e26 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:58.028839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524711e-6ec9-4d24-b013-e2cd34303e26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:15:58.029298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524711e-6ec9-4d24-b013-e2cd34303e26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:58.030852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524711e-6ec9-4d24-b013-e2cd34303e26 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:58.050038 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c18cef-e4d9-4bfb-b588-0cf933a7b5bf None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:58.051640 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 65/130] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:15:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:58.052636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524711e-6ec9-4d24-b013-e2cd34303e26 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:58.093492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d719d-5b64-4165-80ba-9754f0ca885c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:58.093492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d719d-5b64-4165-80ba-9754f0ca885c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:58.093492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d719d-5b64-4165-80ba-9754f0ca885c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:58.093492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d719d-5b64-4165-80ba-9754f0ca885c None admin] PATH_INFO: `/v3/users/f001865a40c84901a03e7897036c3892` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:58.093931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d719d-5b64-4165-80ba-9754f0ca885c None admin] RBAC: Authorizing `identity:get_user(user_id=f001865a40c84901a03e7897036c3892)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:58.095509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d719d-5b64-4165-80ba-9754f0ca885c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:58.097483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 66/131] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:15:57 2022] GET /identity/v3/users/f001865a40c84901a03e7897036c3892 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:58.098439 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d719d-5b64-4165-80ba-9754f0ca885c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:58.212980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d719d-5b64-4165-80ba-9754f0ca885c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:58.250725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29cbf6-0996-4dd1-a2ad-223f2ecfe024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:58.251369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29cbf6-0996-4dd1-a2ad-223f2ecfe024 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:58.251652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29cbf6-0996-4dd1-a2ad-223f2ecfe024 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:58.251927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29cbf6-0996-4dd1-a2ad-223f2ecfe024 None admin] PATH_INFO: `/v3/users/f001865a40c84901a03e7897036c3892` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:58.253078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29cbf6-0996-4dd1-a2ad-223f2ecfe024 None admin] RBAC: Authorizing `identity:get_user(user_id=f001865a40c84901a03e7897036c3892)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:58.255595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:58.260288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29cbf6-0996-4dd1-a2ad-223f2ecfe024 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:58.260288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 67/132] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:15:58 2022] GET /identity/v3/users/f001865a40c84901a03e7897036c3892 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:58.260288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29cbf6-0996-4dd1-a2ad-223f2ecfe024 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:58.274665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:58.291018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524711e-6ec9-4d24-b013-e2cd34303e26 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:58.351631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0ff9f-ea06-4051-add8-2f9316d878d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:58.352697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0ff9f-ea06-4051-add8-2f9316d878d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:58.353215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0ff9f-ea06-4051-add8-2f9316d878d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:58.353705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0ff9f-ea06-4051-add8-2f9316d878d3 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3/users/f615cddbb3154c329188ec780374eff5/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:58.361355 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0ff9f-ea06-4051-add8-2f9316d878d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3, user_id=f615cddbb3154c329188ec780374eff5, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:58.363102 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0ff9f-ea06-4051-add8-2f9316d878d3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:58.381746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29cbf6-0996-4dd1-a2ad-223f2ecfe024 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:58.393283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 66/133] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:15:58 2022] PUT /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3/users/f615cddbb3154c329188ec780374eff5/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:15:58.394547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0ff9f-ea06-4051-add8-2f9316d878d3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:58.434327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab362bdd-9901-4985-814e-71f70e9d6e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:58.434971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab362bdd-9901-4985-814e-71f70e9d6e9f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:58.435253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab362bdd-9901-4985-814e-71f70e9d6e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:58.435529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab362bdd-9901-4985-814e-71f70e9d6e9f None admin] PATH_INFO: `/v3/projects/fe38e48af40b4c28a6808311efb73ac5` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:58.441271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab362bdd-9901-4985-814e-71f70e9d6e9f None admin] RBAC: Authorizing `identity:get_project(project_id=fe38e48af40b4c28a6808311efb73ac5)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:58.448039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab362bdd-9901-4985-814e-71f70e9d6e9f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:58.448039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 68/134] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:15:58 2022] GET /identity/v3/projects/fe38e48af40b4c28a6808311efb73ac5 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:58.451592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab362bdd-9901-4985-814e-71f70e9d6e9f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:58.563739 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0ff9f-ea06-4051-add8-2f9316d878d3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:58.632839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d283897-2bd0-4faa-b5b4-030ece074d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:58.633890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d283897-2bd0-4faa-b5b4-030ece074d0d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:58.634386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d283897-2bd0-4faa-b5b4-030ece074d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:58.634896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d283897-2bd0-4faa-b5b4-030ece074d0d None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:58.636623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d283897-2bd0-4faa-b5b4-030ece074d0d None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:58.638368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d283897-2bd0-4faa-b5b4-030ece074d0d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:58.640345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 67/135] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:15:58 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:15:58.646799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d283897-2bd0-4faa-b5b4-030ece074d0d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:15:58.664060 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:58.684033 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:15:58.690287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab362bdd-9901-4985-814e-71f70e9d6e9f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:58.732926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c7906-8e4d-488f-94dd-8dac62fc81d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:58.733610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7906-8e4d-488f-94dd-8dac62fc81d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:58.733892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7906-8e4d-488f-94dd-8dac62fc81d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:58.734175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c7906-8e4d-488f-94dd-8dac62fc81d7 None admin] PATH_INFO: `/v3/projects/fe38e48af40b4c28a6808311efb73ac5/users/f001865a40c84901a03e7897036c3892/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:58.736260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c7906-8e4d-488f-94dd-8dac62fc81d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe38e48af40b4c28a6808311efb73ac5, user_id=f001865a40c84901a03e7897036c3892, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:58.737882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c7906-8e4d-488f-94dd-8dac62fc81d7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:58.759889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 69/136] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:15:58 2022] PUT /identity/v3/projects/fe38e48af40b4c28a6808311efb73ac5/users/f001865a40c84901a03e7897036c3892/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:15:58.809021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d283897-2bd0-4faa-b5b4-030ece074d0d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:15:58.855873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7214d-31db-4e52-9035-04256ce39785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:15:58.856596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7214d-31db-4e52-9035-04256ce39785 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:58.856882 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7214d-31db-4e52-9035-04256ce39785 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:58.857164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7214d-31db-4e52-9035-04256ce39785 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3/users/f001865a40c84901a03e7897036c3892/roles/6c6c7fa19ee342da915c90895e707620` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:58.859407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7214d-31db-4e52-9035-04256ce39785 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3, user_id=f001865a40c84901a03e7897036c3892, role_id=6c6c7fa19ee342da915c90895e707620)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:15:58.860917 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7214d-31db-4e52-9035-04256ce39785 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:15:58.886387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 68/137] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:15:58 2022] PUT /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3/users/f001865a40c84901a03e7897036c3892/roles/6c6c7fa19ee342da915c90895e707620 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:15:59.347860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f9c79-df4f-4666-bae0-cf6364c8c460 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:59.348516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f9c79-df4f-4666-bae0-cf6364c8c460 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:59.348796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f9c79-df4f-4666-bae0-cf6364c8c460 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:59.350167 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 70/138] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:15:59.364734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba1b5c-96dc-42e0-81d8-d564b85aed2e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:15:59.365046 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba1b5c-96dc-42e0-81d8-d564b85aed2e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:15:59.365309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba1b5c-96dc-42e0-81d8-d564b85aed2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:15:59.394386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b8ba1b5c-96dc-42e0-81d8-d564b85aed2e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:15:59.396340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b8ba1b5c-96dc-42e0-81d8-d564b85aed2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:15:59.484741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 69/139] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:00.019167 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f9c79-df4f-4666-bae0-cf6364c8c460 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:00.120548 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986adba-4ee5-41ee-8f68-dd01c202952c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:00.121572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986adba-4ee5-41ee-8f68-dd01c202952c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:00.122052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986adba-4ee5-41ee-8f68-dd01c202952c None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:00.123627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 70/140] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:00.129321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335db5d-7183-4150-bb6b-cb08c1fe3f7e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:00.129880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335db5d-7183-4150-bb6b-cb08c1fe3f7e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:00.130537 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335db5d-7183-4150-bb6b-cb08c1fe3f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:00.145537 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:00.154225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5335db5d-7183-4150-bb6b-cb08c1fe3f7e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:00.164326 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5335db5d-7183-4150-bb6b-cb08c1fe3f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:00.172839 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f9c79-df4f-4666-bae0-cf6364c8c460 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:00.239618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95d51e-479c-4b13-8d0d-11afd4d4639e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:00.240392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95d51e-479c-4b13-8d0d-11afd4d4639e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:00.240688 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95d51e-479c-4b13-8d0d-11afd4d4639e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:00.240970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95d51e-479c-4b13-8d0d-11afd4d4639e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:00.244747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95d51e-479c-4b13-8d0d-11afd4d4639e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:00.246234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95d51e-479c-4b13-8d0d-11afd4d4639e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:00.249028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 71/141] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:00.250238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834bd88-e3c2-4b9c-aadc-8a82303c3c5a None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:00.250525 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834bd88-e3c2-4b9c-aadc-8a82303c3c5a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:00.250802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834bd88-e3c2-4b9c-aadc-8a82303c3c5a None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:00.252174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 72/142] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:00.259106 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46633c0c-cd40-4418-a4ba-d0364747f6d6 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:00.259397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46633c0c-cd40-4418-a4ba-d0364747f6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:00.259681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46633c0c-cd40-4418-a4ba-d0364747f6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:00.279013 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 71/143] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:00.279687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335db5d-7183-4150-bb6b-cb08c1fe3f7e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:00.288365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:00.302928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-46633c0c-cd40-4418-a4ba-d0364747f6d6 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:00.304201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-46633c0c-cd40-4418-a4ba-d0364747f6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:00.416776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 73/144] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:00.480419 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335db5d-7183-4150-bb6b-cb08c1fe3f7e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:00.533630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72e0b6-4542-4fea-9d90-73dc89a85be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:00.534257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72e0b6-4542-4fea-9d90-73dc89a85be7 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:00.534543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72e0b6-4542-4fea-9d90-73dc89a85be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:00.534823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72e0b6-4542-4fea-9d90-73dc89a85be7 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:00.536088 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72e0b6-4542-4fea-9d90-73dc89a85be7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:00.537563 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72e0b6-4542-4fea-9d90-73dc89a85be7 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:00.551556 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:00.574472 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 72/145] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:16:00 2022] POST /identity/v3/users => generated 283 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:16:00.618001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46633c0c-cd40-4418-a4ba-d0364747f6d6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:00.765274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46633c0c-cd40-4418-a4ba-d0364747f6d6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:00.835610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a623b5-2f03-4928-b7c3-a0716591ef10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:00.836690 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a623b5-2f03-4928-b7c3-a0716591ef10 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:00.837187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a623b5-2f03-4928-b7c3-a0716591ef10 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:00.837664 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a623b5-2f03-4928-b7c3-a0716591ef10 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:00.845362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:00.855955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a623b5-2f03-4928-b7c3-a0716591ef10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e0a623b5-2f03-4928-b7c3-a0716591ef10 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:00.860910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:00.861844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:00.862654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:00.862654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:00.862654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:00.862654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:00.862654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:00.862654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:00.862654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:00.862654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:00.862654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:00.862654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:00.862654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:00.862654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:00.862654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 74/146] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:00.868880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72e0b6-4542-4fea-9d90-73dc89a85be7 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:00.962534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a623b5-2f03-4928-b7c3-a0716591ef10 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:00.993643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72e0b6-4542-4fea-9d90-73dc89a85be7 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:01.040203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ca52f-40bc-41bc-ba7b-a67e4786e68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:01.040764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ca52f-40bc-41bc-ba7b-a67e4786e68c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:01.041049 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ca52f-40bc-41bc-ba7b-a67e4786e68c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:01.041327 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ca52f-40bc-41bc-ba7b-a67e4786e68c None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:01.042019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ca52f-40bc-41bc-ba7b-a67e4786e68c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:01.042327 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ca52f-40bc-41bc-ba7b-a67e4786e68c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:01.043854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ca52f-40bc-41bc-ba7b-a67e4786e68c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:01.055530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 73/147] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:01.065200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ca52f-40bc-41bc-ba7b-a67e4786e68c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:01.100956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a623b5-2f03-4928-b7c3-a0716591ef10 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:01.144230 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:01.147493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aebdf3-5d01-4a7b-99bb-6795f398ca41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:01.148429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aebdf3-5d01-4a7b-99bb-6795f398ca41 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:01.148666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aebdf3-5d01-4a7b-99bb-6795f398ca41 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:01.148958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aebdf3-5d01-4a7b-99bb-6795f398ca41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:01.172023 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aebdf3-5d01-4a7b-99bb-6795f398ca41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d2aebdf3-5d01-4a7b-99bb-6795f398ca41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:01.174194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:01.175049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:01.175847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:01.175847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:01.175847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:01.175847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:01.175847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:01.175847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:01.175847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:01.175847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:01.175847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:01.175847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:01.175847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:01.175847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:01.175847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 75/148] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:16:01.187224 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aebdf3-5d01-4a7b-99bb-6795f398ca41 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:01.217416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ca52f-40bc-41bc-ba7b-a67e4786e68c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:01.272859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecf15f-d57a-40fd-9489-1e57809e35ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:01.273674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecf15f-d57a-40fd-9489-1e57809e35ea None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:01.273961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecf15f-d57a-40fd-9489-1e57809e35ea None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:01.274242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecf15f-d57a-40fd-9489-1e57809e35ea None admin] PATH_INFO: `/v3/users/f152a86ae2354080a31a2daaeec2ef96` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:01.275962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecf15f-d57a-40fd-9489-1e57809e35ea None admin] RBAC: Authorizing `identity:get_user(user_id=f152a86ae2354080a31a2daaeec2ef96)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:01.277781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecf15f-d57a-40fd-9489-1e57809e35ea None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:01.283834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 74/149] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:01 2022] GET /identity/v3/users/f152a86ae2354080a31a2daaeec2ef96 => generated 289 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:01.283834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ec1f6-3a1e-490f-b117-f70e415f69fc None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:01.283834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ec1f6-3a1e-490f-b117-f70e415f69fc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:01.283834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ec1f6-3a1e-490f-b117-f70e415f69fc None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:01.283834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 75/150] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:01.288736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ec1f6-3a1e-490f-b117-f70e415f69fc None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:01.304672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:01.371879 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aebdf3-5d01-4a7b-99bb-6795f398ca41 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:01.425257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d9053-1f93-4105-a219-d46e7cdaffe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:01.427892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d9053-1f93-4105-a219-d46e7cdaffe3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:01.428405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d9053-1f93-4105-a219-d46e7cdaffe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:01.428899 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d9053-1f93-4105-a219-d46e7cdaffe3 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:01.430083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d9053-1f93-4105-a219-d46e7cdaffe3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:01.430595 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d9053-1f93-4105-a219-d46e7cdaffe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:01.433147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d9053-1f93-4105-a219-d46e7cdaffe3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:01.444471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ec1f6-3a1e-490f-b117-f70e415f69fc None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:01.445702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 76/151] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:01.448401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4984bf-2b21-419d-9512-f3011c855599 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:01.448901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4984bf-2b21-419d-9512-f3011c855599 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:01.449391 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4984bf-2b21-419d-9512-f3011c855599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:01.477666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:01.477866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8d4984bf-2b21-419d-9512-f3011c855599 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:01.478964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8d4984bf-2b21-419d-9512-f3011c855599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:01.494323 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aee318-5b07-4bbe-ab32-182276e09176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:01.495362 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aee318-5b07-4bbe-ab32-182276e09176 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:01.495958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aee318-5b07-4bbe-ab32-182276e09176 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:01.496535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aee318-5b07-4bbe-ab32-182276e09176 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:01.497977 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aee318-5b07-4bbe-ab32-182276e09176 None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:01.499878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aee318-5b07-4bbe-ab32-182276e09176 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:01.501875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 76/152] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:01 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:01.502862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aee318-5b07-4bbe-ab32-182276e09176 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:01.559896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 77/153] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:01.560798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4984bf-2b21-419d-9512-f3011c855599 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:01.629655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aee318-5b07-4bbe-ab32-182276e09176 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:01.639745 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:01.670425 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76955235-4e75-4015-b930-660607355971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:01.671514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76955235-4e75-4015-b930-660607355971 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:01.672019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76955235-4e75-4015-b930-660607355971 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:01.672517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76955235-4e75-4015-b930-660607355971 None admin] PATH_INFO: `/v3/users/f001865a40c84901a03e7897036c3892` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:01.673936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76955235-4e75-4015-b930-660607355971 None admin] RBAC: Authorizing `identity:get_user(user_id=f001865a40c84901a03e7897036c3892)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:01.675804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76955235-4e75-4015-b930-660607355971 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:01.678935 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 77/154] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:01 2022] GET /identity/v3/users/f001865a40c84901a03e7897036c3892 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:01.684900 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76955235-4e75-4015-b930-660607355971 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:01.751357 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:01.760039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4984bf-2b21-419d-9512-f3011c855599 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:01.810733 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76955235-4e75-4015-b930-660607355971 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:01.850853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebe80c-d8f9-4495-8257-ba050c4ba3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:01.851531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebe80c-d8f9-4495-8257-ba050c4ba3e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:01.851807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebe80c-d8f9-4495-8257-ba050c4ba3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:01.852104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebe80c-d8f9-4495-8257-ba050c4ba3e6 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3/users/f152a86ae2354080a31a2daaeec2ef96/roles/8cf91ce8e7dd42aaa2708185cb13a6db` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:01.858111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e22a4-991d-4fc4-9004-15140145ce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:01.858243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebe80c-d8f9-4495-8257-ba050c4ba3e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3, user_id=f152a86ae2354080a31a2daaeec2ef96, role_id=8cf91ce8e7dd42aaa2708185cb13a6db)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:01.859102 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e22a4-991d-4fc4-9004-15140145ce16 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:01.859637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebe80c-d8f9-4495-8257-ba050c4ba3e6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:01.859895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e22a4-991d-4fc4-9004-15140145ce16 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:01.860378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e22a4-991d-4fc4-9004-15140145ce16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:01.884066 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 78/155] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:16:01 2022] PUT /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3/users/f152a86ae2354080a31a2daaeec2ef96/roles/8cf91ce8e7dd42aaa2708185cb13a6db => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:16:01.967689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e22a4-991d-4fc4-9004-15140145ce16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:01.969228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e22a4-991d-4fc4-9004-15140145ce16 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:01.973785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebe80c-d8f9-4495-8257-ba050c4ba3e6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:02.053442 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:02.112750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 78/156] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 05:16:02.119527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e22a4-991d-4fc4-9004-15140145ce16 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:02.150174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebe80c-d8f9-4495-8257-ba050c4ba3e6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:02.165261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:02.212039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970c2ac-2563-466a-9694-22917777326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:02.212039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970c2ac-2563-466a-9694-22917777326f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:02.212039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970c2ac-2563-466a-9694-22917777326f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:02.212039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970c2ac-2563-466a-9694-22917777326f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:02.227414 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970c2ac-2563-466a-9694-22917777326f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-3970c2ac-2563-466a-9694-22917777326f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:02.236482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:02.237680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:02.239135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:02.239135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:02.239135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:02.239135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:02.239135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:02.239135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:02.239135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:02.239135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:02.239135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:02.239135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:02.239135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:02.239135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:02.239135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 79/157] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:16:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:02.245834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970c2ac-2563-466a-9694-22917777326f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:02.291789 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e22a4-991d-4fc4-9004-15140145ce16 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:02.341809 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd3719-53f6-49e3-a845-ed2b92c79725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:02.342723 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd3719-53f6-49e3-a845-ed2b92c79725 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:02.343154 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd3719-53f6-49e3-a845-ed2b92c79725 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:02.343575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd3719-53f6-49e3-a845-ed2b92c79725 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:02.348750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd3719-53f6-49e3-a845-ed2b92c79725 None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:02.350460 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd3719-53f6-49e3-a845-ed2b92c79725 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:02.352361 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 79/158] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:02 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:02.358358 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd3719-53f6-49e3-a845-ed2b92c79725 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:02.401855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970c2ac-2563-466a-9694-22917777326f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:02.404152 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:02.454070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29008b-eff0-46a2-b60a-1ef74e05a1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:02.454696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29008b-eff0-46a2-b60a-1ef74e05a1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:02.454978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29008b-eff0-46a2-b60a-1ef74e05a1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:02.455250 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29008b-eff0-46a2-b60a-1ef74e05a1a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:02.455934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29008b-eff0-46a2-b60a-1ef74e05a1a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:02.456267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29008b-eff0-46a2-b60a-1ef74e05a1a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:02.457715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29008b-eff0-46a2-b60a-1ef74e05a1a6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:02.474209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 80/159] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:02.477124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98243c1-ff67-4461-94af-84a4c0e734f2 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:02.478317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98243c1-ff67-4461-94af-84a4c0e734f2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:02.478984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98243c1-ff67-4461-94af-84a4c0e734f2 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:02.481966 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 81/160] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:02.483956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98243c1-ff67-4461-94af-84a4c0e734f2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:02.536943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd3719-53f6-49e3-a845-ed2b92c79725 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:02.555974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:02.638141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98243c1-ff67-4461-94af-84a4c0e734f2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:02.648730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095766d9-25a4-4db1-bb08-5d515e286714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:02.649355 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095766d9-25a4-4db1-bb08-5d515e286714 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:02.649641 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095766d9-25a4-4db1-bb08-5d515e286714 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:02.649915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095766d9-25a4-4db1-bb08-5d515e286714 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:02.650660 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095766d9-25a4-4db1-bb08-5d515e286714 None admin] RBAC: Adding query filter params (role.id=d9f6babaca2d494c9a04f0aca1271af8, scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3, user.id=f001865a40c84901a03e7897036c3892) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:02.650967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095766d9-25a4-4db1-bb08-5d515e286714 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:02.652545 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095766d9-25a4-4db1-bb08-5d515e286714 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:02.666783 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 80/161] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:02 2022] GET /identity/v3/role_assignments?user.id=f001865a40c84901a03e7897036c3892&scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3&role.id=d9f6babaca2d494c9a04f0aca1271af8 => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:02.672022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023a97e-2d1e-4900-aaaf-ac859b8b4484 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:02.672022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023a97e-2d1e-4900-aaaf-ac859b8b4484 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:02.672022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023a97e-2d1e-4900-aaaf-ac859b8b4484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:02.684796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f4259-d9e4-439c-b0c7-058dd5dd186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:02.685950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f4259-d9e4-439c-b0c7-058dd5dd186f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:02.686436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f4259-d9e4-439c-b0c7-058dd5dd186f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:02.687084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f4259-d9e4-439c-b0c7-058dd5dd186f None admin] PATH_INFO: `/v3/users/f615cddbb3154c329188ec780374eff5` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:02.688633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f4259-d9e4-439c-b0c7-058dd5dd186f None admin] RBAC: Authorizing `identity:get_user(user_id=f615cddbb3154c329188ec780374eff5)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:02.690375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f4259-d9e4-439c-b0c7-058dd5dd186f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:02.693245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 82/162] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:02 2022] GET /identity/v3/users/f615cddbb3154c329188ec780374eff5 => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:02.699698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f4259-d9e4-439c-b0c7-058dd5dd186f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:02.720491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-1023a97e-2d1e-4900-aaaf-ac859b8b4484 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:02.722585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1023a97e-2d1e-4900-aaaf-ac859b8b4484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:02.868469 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 81/163] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:02.923476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:02.928303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f4259-d9e4-439c-b0c7-058dd5dd186f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:02.978750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11408a2e-d5d5-4c8f-827b-8f5b5756e62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:02.979383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11408a2e-d5d5-4c8f-827b-8f5b5756e62e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:02.979667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11408a2e-d5d5-4c8f-827b-8f5b5756e62e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:02.979944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11408a2e-d5d5-4c8f-827b-8f5b5756e62e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:03.076111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11408a2e-d5d5-4c8f-827b-8f5b5756e62e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:03.077630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11408a2e-d5d5-4c8f-827b-8f5b5756e62e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:03.209547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:03.214516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 83/164] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 05:16:03.221522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023a97e-2d1e-4900-aaaf-ac859b8b4484 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:03.259461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:03.362640 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023a97e-2d1e-4900-aaaf-ac859b8b4484 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:03.380613 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11408a2e-d5d5-4c8f-827b-8f5b5756e62e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:03.413391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f66de-1fb6-4ce3-b207-62844d0f7ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:03.414053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f66de-1fb6-4ce3-b207-62844d0f7ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:03.414336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f66de-1fb6-4ce3-b207-62844d0f7ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:03.414612 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f66de-1fb6-4ce3-b207-62844d0f7ec8 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:03.415792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f66de-1fb6-4ce3-b207-62844d0f7ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:03.417426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f66de-1fb6-4ce3-b207-62844d0f7ec8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:03.419147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 82/165] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:03 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:03.425257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f66de-1fb6-4ce3-b207-62844d0f7ec8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:03.547302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11408a2e-d5d5-4c8f-827b-8f5b5756e62e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:03.587361 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad93d4-5adc-4369-a37e-f3d153b6f407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:03.588377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad93d4-5adc-4369-a37e-f3d153b6f407 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:03.589278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad93d4-5adc-4369-a37e-f3d153b6f407 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:03.589756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad93d4-5adc-4369-a37e-f3d153b6f407 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:03.601447 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad93d4-5adc-4369-a37e-f3d153b6f407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d0ad93d4-5adc-4369-a37e-f3d153b6f407 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:03.607802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:03.608758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:03.609708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:03.609708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:03.609708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:03.609708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:03.609708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:03.609708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:03.609708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:03.609708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:03.609708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:03.609708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:03.609708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:03.609708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:03.611963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 84/166] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:16:03.614758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da86e9-817a-4474-98bf-7a0f0236226d None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:03.615384 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da86e9-817a-4474-98bf-7a0f0236226d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:03.616338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da86e9-817a-4474-98bf-7a0f0236226d None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:03.617899 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 85/167] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:03.618796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da86e9-817a-4474-98bf-7a0f0236226d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:03.656798 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:03.704509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:03.718297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f66de-1fb6-4ce3-b207-62844d0f7ec8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:03.740855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da86e9-817a-4474-98bf-7a0f0236226d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:03.781593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1fe5a-b25c-459a-a36d-51b8b0b2de10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:03.782692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1fe5a-b25c-459a-a36d-51b8b0b2de10 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:03.783168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1fe5a-b25c-459a-a36d-51b8b0b2de10 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:03.783761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1fe5a-b25c-459a-a36d-51b8b0b2de10 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:03.784699 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1fe5a-b25c-459a-a36d-51b8b0b2de10 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:03.785258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1fe5a-b25c-459a-a36d-51b8b0b2de10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:03.787855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1fe5a-b25c-459a-a36d-51b8b0b2de10 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:03.797153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 86/168] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:03.798728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907bd94-98fc-4cda-96ca-ce23df385668 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:03.799313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907bd94-98fc-4cda-96ca-ce23df385668 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:03.799805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907bd94-98fc-4cda-96ca-ce23df385668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:03.816796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb663db-1193-4154-af78-526cb61e93c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:03.817576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb663db-1193-4154-af78-526cb61e93c6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:03.817856 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb663db-1193-4154-af78-526cb61e93c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:03.818132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb663db-1193-4154-af78-526cb61e93c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:03.818876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb663db-1193-4154-af78-526cb61e93c6 None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3, user.id=f615cddbb3154c329188ec780374eff5) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:03.819183 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb663db-1193-4154-af78-526cb61e93c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:03.820677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb663db-1193-4154-af78-526cb61e93c6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:03.821442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f907bd94-98fc-4cda-96ca-ce23df385668 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:03.822994 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f907bd94-98fc-4cda-96ca-ce23df385668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:03.833035 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 83/169] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:03 2022] GET /identity/v3/role_assignments?user.id=f615cddbb3154c329188ec780374eff5&scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 608 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:03.833986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb663db-1193-4154-af78-526cb61e93c6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:03.899196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 87/170] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:03.986292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb663db-1193-4154-af78-526cb61e93c6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:04.032651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8424994-5bdd-4781-a62c-49335fc7bc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:04.033658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8424994-5bdd-4781-a62c-49335fc7bc75 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:04.034178 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8424994-5bdd-4781-a62c-49335fc7bc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:04.034657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8424994-5bdd-4781-a62c-49335fc7bc75 None admin] PATH_INFO: `/v3/users/f001865a40c84901a03e7897036c3892` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:04.040705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8424994-5bdd-4781-a62c-49335fc7bc75 None admin] RBAC: Authorizing `identity:get_user(user_id=f001865a40c84901a03e7897036c3892)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:04.042442 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8424994-5bdd-4781-a62c-49335fc7bc75 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:04.044583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 84/171] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:03 2022] GET /identity/v3/users/f001865a40c84901a03e7897036c3892 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:04.049840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907bd94-98fc-4cda-96ca-ce23df385668 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:04.128214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:04.186705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907bd94-98fc-4cda-96ca-ce23df385668 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:04.233865 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef1280-ea9e-47c5-ab81-a037a00d9bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:04.234492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef1280-ea9e-47c5-ab81-a037a00d9bab None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:04.234779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef1280-ea9e-47c5-ab81-a037a00d9bab None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:04.235052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef1280-ea9e-47c5-ab81-a037a00d9bab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:04.331465 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef1280-ea9e-47c5-ab81-a037a00d9bab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:04.333519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef1280-ea9e-47c5-ab81-a037a00d9bab None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:04.405110 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:04.461339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 88/172] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:04.467356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8424994-5bdd-4781-a62c-49335fc7bc75 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:04.500662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:04.599653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef1280-ea9e-47c5-ab81-a037a00d9bab None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:04.605870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8424994-5bdd-4781-a62c-49335fc7bc75 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:04.651550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c8fd1-9bd4-4017-8697-db1d39896b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:04.652231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c8fd1-9bd4-4017-8697-db1d39896b0d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:04.652518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c8fd1-9bd4-4017-8697-db1d39896b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:04.652794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c8fd1-9bd4-4017-8697-db1d39896b0d None admin] PATH_INFO: `/v3/projects/fe38e48af40b4c28a6808311efb73ac5` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:04.656707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c8fd1-9bd4-4017-8697-db1d39896b0d None admin] RBAC: Authorizing `identity:get_project(project_id=fe38e48af40b4c28a6808311efb73ac5)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:04.658199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c8fd1-9bd4-4017-8697-db1d39896b0d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:04.660462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 85/173] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:04 2022] GET /identity/v3/projects/fe38e48af40b4c28a6808311efb73ac5 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:04.668262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c8fd1-9bd4-4017-8697-db1d39896b0d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:04.813411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c8fd1-9bd4-4017-8697-db1d39896b0d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:04.816342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:04.871461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8e0c7-09e3-4e57-b7b1-180503f6f929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:04.872446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8e0c7-09e3-4e57-b7b1-180503f6f929 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:04.872936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8e0c7-09e3-4e57-b7b1-180503f6f929 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:04.873435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8e0c7-09e3-4e57-b7b1-180503f6f929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:04.874372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8e0c7-09e3-4e57-b7b1-180503f6f929 None admin] RBAC: Adding query filter params (role.id=d9f6babaca2d494c9a04f0aca1271af8, scope.project.id=fe38e48af40b4c28a6808311efb73ac5, user.id=f001865a40c84901a03e7897036c3892) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:04.874895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8e0c7-09e3-4e57-b7b1-180503f6f929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:04.876619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8e0c7-09e3-4e57-b7b1-180503f6f929 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:04.889022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 86/174] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:04 2022] GET /identity/v3/role_assignments?user.id=f001865a40c84901a03e7897036c3892&scope.project.id=fe38e48af40b4c28a6808311efb73ac5&role.id=d9f6babaca2d494c9a04f0aca1271af8 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:04.927331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef1280-ea9e-47c5-ab81-a037a00d9bab None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:04.982585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d64e9-3c2f-495e-bd91-c8e6d924314e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:04.983229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d64e9-3c2f-495e-bd91-c8e6d924314e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:04.983508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d64e9-3c2f-495e-bd91-c8e6d924314e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:04.983779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d64e9-3c2f-495e-bd91-c8e6d924314e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:05.000922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d64e9-3c2f-495e-bd91-c8e6d924314e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-0f3d64e9-3c2f-495e-bd91-c8e6d924314e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:05.006685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:05.007628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:05.008446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:05.008446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:05.008446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:05.008446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:05.008446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:05.008446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:05.008446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:05.008446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:05.008446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:05.008446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:05.008446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:05.008446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:05.012183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 89/175] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:16:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:05.013956 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8e0c7-09e3-4e57-b7b1-180503f6f929 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:05.242639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8e0c7-09e3-4e57-b7b1-180503f6f929 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:05.297618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ca08f-49a2-4b85-9b33-633949b99993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:05.298248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ca08f-49a2-4b85-9b33-633949b99993 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:05.298530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ca08f-49a2-4b85-9b33-633949b99993 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:05.298805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ca08f-49a2-4b85-9b33-633949b99993 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:05.299481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ca08f-49a2-4b85-9b33-633949b99993 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:05.299785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ca08f-49a2-4b85-9b33-633949b99993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:05.301765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ca08f-49a2-4b85-9b33-633949b99993 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:05.315365 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 87/176] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:05.321366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d64e9-3c2f-495e-bd91-c8e6d924314e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:05.340029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:05.389840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6973570-7591-405c-afc1-028ad7c66975 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:05.390161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6973570-7591-405c-afc1-028ad7c66975 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:05.390394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6973570-7591-405c-afc1-028ad7c66975 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:05.391851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 88/177] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:05.400732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f7822-1a31-467f-a6e3-9f50dc5ea32d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:05.401023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f7822-1a31-467f-a6e3-9f50dc5ea32d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:05.401283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f7822-1a31-467f-a6e3-9f50dc5ea32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:05.428855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0b1f7822-1a31-467f-a6e3-9f50dc5ea32d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:05.430076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0b1f7822-1a31-467f-a6e3-9f50dc5ea32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:05.444372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:05.505316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d64e9-3c2f-495e-bd91-c8e6d924314e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:05.541984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb8812-3616-431b-bb41-a185ab6ed091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:05.542629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb8812-3616-431b-bb41-a185ab6ed091 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:05.542905 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb8812-3616-431b-bb41-a185ab6ed091 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:05.543172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb8812-3616-431b-bb41-a185ab6ed091 None admin] PATH_INFO: `/v3/users/f001865a40c84901a03e7897036c3892` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:05.544349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb8812-3616-431b-bb41-a185ab6ed091 None admin] RBAC: Authorizing `identity:get_user(user_id=f001865a40c84901a03e7897036c3892)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:05.546009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb8812-3616-431b-bb41-a185ab6ed091 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:05.547698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 90/179] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:05 2022] GET /identity/v3/users/f001865a40c84901a03e7897036c3892 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:05.547812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 89/179] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:05.549795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959eac6-060d-4254-931c-85426e63205e None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:05.550074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959eac6-060d-4254-931c-85426e63205e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:05.550342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959eac6-060d-4254-931c-85426e63205e None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:05.552044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 91/180] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:05.557625 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a212c-997b-42b5-936a-9079360e5279 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:05.557884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959eac6-060d-4254-931c-85426e63205e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:05.557988 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a212c-997b-42b5-936a-9079360e5279 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:05.558171 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a212c-997b-42b5-936a-9079360e5279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:05.596976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b27a212c-997b-42b5-936a-9079360e5279 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:05.598690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b27a212c-997b-42b5-936a-9079360e5279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:05.648016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:05.672419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959eac6-060d-4254-931c-85426e63205e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:05.695988 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 90/181] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:05.736651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f90f48-7995-4eb0-8454-81e6dd001cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:05.737272 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f90f48-7995-4eb0-8454-81e6dd001cfd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:05.737558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f90f48-7995-4eb0-8454-81e6dd001cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:05.737918 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f90f48-7995-4eb0-8454-81e6dd001cfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:05.757518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3e053-3414-4868-a014-a1d6cec5faa2 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:05.757808 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3e053-3414-4868-a014-a1d6cec5faa2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:05.757988 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3e053-3414-4868-a014-a1d6cec5faa2 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:05.759349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 91/182] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:05.765759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8b0bc-0e86-4a23-ae9c-697cdca8e1fb None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:05.766045 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8b0bc-0e86-4a23-ae9c-697cdca8e1fb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:05.766321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8b0bc-0e86-4a23-ae9c-697cdca8e1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:05.845169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-93f8b0bc-0e86-4a23-ae9c-697cdca8e1fb None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:05.848023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-93f8b0bc-0e86-4a23-ae9c-697cdca8e1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:05.903434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f90f48-7995-4eb0-8454-81e6dd001cfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:05.905012 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f90f48-7995-4eb0-8454-81e6dd001cfd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:05.974775 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 92/183] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:05.980883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8b0bc-0e86-4a23-ae9c-697cdca8e1fb None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:05.988922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:06.034285 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:06.063882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 92/184] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 05:16:06.065399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc8b3d-5d5e-4755-a4f5-612e51fa0ed6 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:06.065750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc8b3d-5d5e-4755-a4f5-612e51fa0ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:06.066108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc8b3d-5d5e-4755-a4f5-612e51fa0ed6 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:06.068259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 93/185] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:06.069170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc8b3d-5d5e-4755-a4f5-612e51fa0ed6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:06.137252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8b0bc-0e86-4a23-ae9c-697cdca8e1fb None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:06.188241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8105b-4646-4fed-bf9b-4d2f88fcc367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:06.188608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc8b3d-5d5e-4755-a4f5-612e51fa0ed6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:06.196022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8105b-4646-4fed-bf9b-4d2f88fcc367 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:06.196022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8105b-4646-4fed-bf9b-4d2f88fcc367 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:06.196022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8105b-4646-4fed-bf9b-4d2f88fcc367 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:06.203534 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8105b-4646-4fed-bf9b-4d2f88fcc367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-8de8105b-4646-4fed-bf9b-4d2f88fcc367 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:06.209888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:06.210975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:06.212011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:06.212011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:06.212011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:06.212011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:06.212011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:06.212011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:06.212011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:06.212011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:06.212011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:06.212011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:06.212011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:06.212011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:06.212011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 93/186] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:16:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:06.212840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8105b-4646-4fed-bf9b-4d2f88fcc367 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:06.233815 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09397f-5e6c-494f-9108-43a58b10fc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:06.234456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09397f-5e6c-494f-9108-43a58b10fc73 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:06.234734 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09397f-5e6c-494f-9108-43a58b10fc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:06.235000 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09397f-5e6c-494f-9108-43a58b10fc73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:06.245415 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09397f-5e6c-494f-9108-43a58b10fc73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-dc09397f-5e6c-494f-9108-43a58b10fc73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:06.250253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:06.251324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 94/187] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:16:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:06.252849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7e869-d1d5-48fa-85ad-203d5247cf07 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:06.254348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7e869-d1d5-48fa-85ad-203d5247cf07 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:06.254348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7e869-d1d5-48fa-85ad-203d5247cf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:06.273483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4bf7e869-d1d5-48fa-85ad-203d5247cf07 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:06.274676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4bf7e869-d1d5-48fa-85ad-203d5247cf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:06.290677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:06.307004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:06.329112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8105b-4646-4fed-bf9b-4d2f88fcc367 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:06.344645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 95/188] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:06.345506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7e869-d1d5-48fa-85ad-203d5247cf07 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:06.374038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a7d81-3760-4acc-b82c-f8eca1cb5283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:06.374667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a7d81-3760-4acc-b82c-f8eca1cb5283 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:06.374944 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a7d81-3760-4acc-b82c-f8eca1cb5283 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:06.375212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a7d81-3760-4acc-b82c-f8eca1cb5283 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:06.376347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a7d81-3760-4acc-b82c-f8eca1cb5283 None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:06.377809 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a7d81-3760-4acc-b82c-f8eca1cb5283 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:06.380413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 94/189] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:06 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:06.381170 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a7d81-3760-4acc-b82c-f8eca1cb5283 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:06.455346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7e869-d1d5-48fa-85ad-203d5247cf07 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:06.486490 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a7d81-3760-4acc-b82c-f8eca1cb5283 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:06.495945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85297dd1-b27b-4201-ad42-900708d6763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:06.496591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85297dd1-b27b-4201-ad42-900708d6763f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:06.496970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85297dd1-b27b-4201-ad42-900708d6763f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:06.497243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85297dd1-b27b-4201-ad42-900708d6763f None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:06.497904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85297dd1-b27b-4201-ad42-900708d6763f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:06.498225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85297dd1-b27b-4201-ad42-900708d6763f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:06.499736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85297dd1-b27b-4201-ad42-900708d6763f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:06.512849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 96/190] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:06.512849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85297dd1-b27b-4201-ad42-900708d6763f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:06.523154 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:06.530561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f531b-b5fc-492d-961b-b182a00d52dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:06.531594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f531b-b5fc-492d-961b-b182a00d52dd None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:06.531880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f531b-b5fc-492d-961b-b182a00d52dd None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:06.532182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f531b-b5fc-492d-961b-b182a00d52dd None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:06.533003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f531b-b5fc-492d-961b-b182a00d52dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:06.533306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f531b-b5fc-492d-961b-b182a00d52dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:06.534693 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f531b-b5fc-492d-961b-b182a00d52dd None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:06.541173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:06.545131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 95/191] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:06.550643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f531b-b5fc-492d-961b-b182a00d52dd None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:06.615960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85297dd1-b27b-4201-ad42-900708d6763f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:06.649558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6e732-add6-4c1a-8dbc-131e21ec9d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:06.650175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6e732-add6-4c1a-8dbc-131e21ec9d22 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:06.650451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6e732-add6-4c1a-8dbc-131e21ec9d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:06.650718 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6e732-add6-4c1a-8dbc-131e21ec9d22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:06.651808 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6e732-add6-4c1a-8dbc-131e21ec9d22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:06.653277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6e732-add6-4c1a-8dbc-131e21ec9d22 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:06.655020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 97/192] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:06.655577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6e732-add6-4c1a-8dbc-131e21ec9d22 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:06.655743 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f531b-b5fc-492d-961b-b182a00d52dd None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:06.691436 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee8483-06c3-488d-82ae-3a0d21c1f238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:06.692071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee8483-06c3-488d-82ae-3a0d21c1f238 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:06.692360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee8483-06c3-488d-82ae-3a0d21c1f238 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:06.692635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee8483-06c3-488d-82ae-3a0d21c1f238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:06.693358 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee8483-06c3-488d-82ae-3a0d21c1f238 None admin] RBAC: Adding query filter params (role.id=6c6c7fa19ee342da915c90895e707620, scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3, user.id=f001865a40c84901a03e7897036c3892) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:06.693758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee8483-06c3-488d-82ae-3a0d21c1f238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:06.695206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee8483-06c3-488d-82ae-3a0d21c1f238 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:06.700887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 96/193] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:06 2022] GET /identity/v3/role_assignments?user.id=f001865a40c84901a03e7897036c3892&scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3&role.id=6c6c7fa19ee342da915c90895e707620 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:06.701851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee8483-06c3-488d-82ae-3a0d21c1f238 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:06.764261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:06.770026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6e732-add6-4c1a-8dbc-131e21ec9d22 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:06.773833 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:06.812042 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ade09-c6d0-4c22-9a7f-e6ec7362ce40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:06.812693 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ade09-c6d0-4c22-9a7f-e6ec7362ce40 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:06.812976 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ade09-c6d0-4c22-9a7f-e6ec7362ce40 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:06.813250 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ade09-c6d0-4c22-9a7f-e6ec7362ce40 None admin] PATH_INFO: `/v3/users/8b5a7b7bd6b54fcf821970d4d9ada6a4` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:06.829883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ade09-c6d0-4c22-9a7f-e6ec7362ce40 None admin] RBAC: Authorizing `identity:get_user(user_id=8b5a7b7bd6b54fcf821970d4d9ada6a4)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:06.830263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee8483-06c3-488d-82ae-3a0d21c1f238 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:06.831417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ade09-c6d0-4c22-9a7f-e6ec7362ce40 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:06.833987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 98/194] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:06 2022] GET /identity/v3/users/8b5a7b7bd6b54fcf821970d4d9ada6a4 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:06.834332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ade09-c6d0-4c22-9a7f-e6ec7362ce40 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:06.873407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80c3de-5933-4012-9324-77fd03acf155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:06.876407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c3de-5933-4012-9324-77fd03acf155 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:06.876407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c3de-5933-4012-9324-77fd03acf155 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:06.876407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c3de-5933-4012-9324-77fd03acf155 None admin] PATH_INFO: `/v3/users/8b5a7b7bd6b54fcf821970d4d9ada6a4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:06.877023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80c3de-5933-4012-9324-77fd03acf155 None admin] RBAC: Authorizing `identity:get_user(user_id=8b5a7b7bd6b54fcf821970d4d9ada6a4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:06.880552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80c3de-5933-4012-9324-77fd03acf155 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:06.882175 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 97/195] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:06 2022] GET /identity/v3/users/8b5a7b7bd6b54fcf821970d4d9ada6a4 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:06.886962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80c3de-5933-4012-9324-77fd03acf155 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:06.953388 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ade09-c6d0-4c22-9a7f-e6ec7362ce40 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:06.987043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19fe54-5cd7-496b-be65-da912f656554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:06.987678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19fe54-5cd7-496b-be65-da912f656554 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:06.987955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19fe54-5cd7-496b-be65-da912f656554 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:06.988916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19fe54-5cd7-496b-be65-da912f656554 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:06.998414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80c3de-5933-4012-9324-77fd03acf155 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:06.999451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19fe54-5cd7-496b-be65-da912f656554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-4a19fe54-5cd7-496b-be65-da912f656554 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:07.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:07.005512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:07.006489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:07.006489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:07.006489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:07.006489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:07.006489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:07.006489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:07.006489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:07.006489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:07.006489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:07.006489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:07.006489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:07.006489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:07.006489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 99/196] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:07.008243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19fe54-5cd7-496b-be65-da912f656554 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:07.024517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:07.025735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:07.034148 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcbdc3-cab1-4a5e-8dda-b1fb0c3f77ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:07.034758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcbdc3-cab1-4a5e-8dda-b1fb0c3f77ca None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:07.035041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcbdc3-cab1-4a5e-8dda-b1fb0c3f77ca None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:07.035846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcbdc3-cab1-4a5e-8dda-b1fb0c3f77ca None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:07.036539 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcbdc3-cab1-4a5e-8dda-b1fb0c3f77ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:07.037986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcbdc3-cab1-4a5e-8dda-b1fb0c3f77ca None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:07.058919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 98/197] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:16:06 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:16:07.061092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcbdc3-cab1-4a5e-8dda-b1fb0c3f77ca None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:07.124247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19fe54-5cd7-496b-be65-da912f656554 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:07.165482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c1eef-685d-4b5a-9057-f83bb74e45af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:07.166110 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c1eef-685d-4b5a-9057-f83bb74e45af None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:07.166421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c1eef-685d-4b5a-9057-f83bb74e45af None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:07.166666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c1eef-685d-4b5a-9057-f83bb74e45af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:07.177973 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcbdc3-cab1-4a5e-8dda-b1fb0c3f77ca None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:07.221134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c259bf-5445-493b-babf-5e20939e0f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:07.221757 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c259bf-5445-493b-babf-5e20939e0f8c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:07.222034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c259bf-5445-493b-babf-5e20939e0f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:07.222309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c259bf-5445-493b-babf-5e20939e0f8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:07.245693 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c1eef-685d-4b5a-9057-f83bb74e45af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:07.247563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c1eef-685d-4b5a-9057-f83bb74e45af None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:07.266063 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:07.286417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:07.312830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c259bf-5445-493b-babf-5e20939e0f8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:07.314248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c259bf-5445-493b-babf-5e20939e0f8c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:07.370192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 100/198] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:07.371288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c1eef-685d-4b5a-9057-f83bb74e45af None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:07.445060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 99/199] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:07.446274 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c259bf-5445-493b-babf-5e20939e0f8c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:07.488586 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c1eef-685d-4b5a-9057-f83bb74e45af None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:07.508166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:07.528487 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6102d-2a4c-4bc8-9095-61c82e26a947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:07.529481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6102d-2a4c-4bc8-9095-61c82e26a947 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:07.530143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6102d-2a4c-4bc8-9095-61c82e26a947 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:07.530612 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6102d-2a4c-4bc8-9095-61c82e26a947 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:07.532027 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6102d-2a4c-4bc8-9095-61c82e26a947 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:07.532587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6102d-2a4c-4bc8-9095-61c82e26a947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:07.534140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c6102d-2a4c-4bc8-9095-61c82e26a947 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:07.544729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 101/200] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:07.545873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6102d-2a4c-4bc8-9095-61c82e26a947 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:07.564047 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:07.589142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c259bf-5445-493b-babf-5e20939e0f8c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:07.628764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094391e-acf6-4849-b48a-0d84eed7a48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:07.630048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094391e-acf6-4849-b48a-0d84eed7a48c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:07.630497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094391e-acf6-4849-b48a-0d84eed7a48c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:07.630949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094391e-acf6-4849-b48a-0d84eed7a48c None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:07.632362 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094391e-acf6-4849-b48a-0d84eed7a48c None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:07.634064 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094391e-acf6-4849-b48a-0d84eed7a48c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:07.640123 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 100/201] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:07 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:07.641309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094391e-acf6-4849-b48a-0d84eed7a48c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:07.672461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6102d-2a4c-4bc8-9095-61c82e26a947 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:07.711567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d20dc-ecf4-42b5-95b8-fa9598cee844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:07.712362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d20dc-ecf4-42b5-95b8-fa9598cee844 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:07.712716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d20dc-ecf4-42b5-95b8-fa9598cee844 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:07.713065 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d20dc-ecf4-42b5-95b8-fa9598cee844 None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:07.714337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d20dc-ecf4-42b5-95b8-fa9598cee844 None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:07.715868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d20dc-ecf4-42b5-95b8-fa9598cee844 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:07.717686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 102/202] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:07 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:07.718642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d20dc-ecf4-42b5-95b8-fa9598cee844 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:07.775223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094391e-acf6-4849-b48a-0d84eed7a48c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:07.784951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:07.817505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132dbab-212e-4914-b2e8-16bbd7b2f73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:07.818668 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132dbab-212e-4914-b2e8-16bbd7b2f73d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:07.819064 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132dbab-212e-4914-b2e8-16bbd7b2f73d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:07.819446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132dbab-212e-4914-b2e8-16bbd7b2f73d None admin] PATH_INFO: `/v3/users/f152a86ae2354080a31a2daaeec2ef96` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:07.821321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132dbab-212e-4914-b2e8-16bbd7b2f73d None admin] RBAC: Authorizing `identity:get_user(user_id=f152a86ae2354080a31a2daaeec2ef96)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:07.823528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132dbab-212e-4914-b2e8-16bbd7b2f73d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:07.825710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 101/203] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:07 2022] GET /identity/v3/users/f152a86ae2354080a31a2daaeec2ef96 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:07.828838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132dbab-212e-4914-b2e8-16bbd7b2f73d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:07.848359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d20dc-ecf4-42b5-95b8-fa9598cee844 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:07.856606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:07.891246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb8af0-8e49-4f8b-8c06-1a03d3f6377c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:07.892132 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb8af0-8e49-4f8b-8c06-1a03d3f6377c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:07.892756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb8af0-8e49-4f8b-8c06-1a03d3f6377c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:07.893188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb8af0-8e49-4f8b-8c06-1a03d3f6377c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:07.894160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb8af0-8e49-4f8b-8c06-1a03d3f6377c None admin] RBAC: Adding query filter params (role.id=6c6c7fa19ee342da915c90895e707620, scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17, user.id=8b5a7b7bd6b54fcf821970d4d9ada6a4) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:07.895168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb8af0-8e49-4f8b-8c06-1a03d3f6377c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:07.897458 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb8af0-8e49-4f8b-8c06-1a03d3f6377c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:07.904610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 103/204] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:07 2022] GET /identity/v3/role_assignments?user.id=8b5a7b7bd6b54fcf821970d4d9ada6a4&scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17&role.id=6c6c7fa19ee342da915c90895e707620 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:07.912273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb8af0-8e49-4f8b-8c06-1a03d3f6377c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:08.009744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132dbab-212e-4914-b2e8-16bbd7b2f73d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:08.081411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49079f7-3fff-443e-ab20-3a293ef20bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:08.082472 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49079f7-3fff-443e-ab20-3a293ef20bed None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:08.082754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49079f7-3fff-443e-ab20-3a293ef20bed None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:08.083025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49079f7-3fff-443e-ab20-3a293ef20bed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:08.083768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49079f7-3fff-443e-ab20-3a293ef20bed None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17, user.id=8b5a7b7bd6b54fcf821970d4d9ada6a4) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:08.084112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49079f7-3fff-443e-ab20-3a293ef20bed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:08.086331 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49079f7-3fff-443e-ab20-3a293ef20bed None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:08.093027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 102/205] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:07 2022] GET /identity/v3/role_assignments?user.id=8b5a7b7bd6b54fcf821970d4d9ada6a4&scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:08.131476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb8af0-8e49-4f8b-8c06-1a03d3f6377c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:08.147293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:08.174862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aae9d5-55cb-479e-b635-50ec66c08d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:08.176553 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aae9d5-55cb-479e-b635-50ec66c08d41 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:08.177038 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aae9d5-55cb-479e-b635-50ec66c08d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:08.177536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aae9d5-55cb-479e-b635-50ec66c08d41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:08.276956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aae9d5-55cb-479e-b635-50ec66c08d41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:08.278867 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aae9d5-55cb-479e-b635-50ec66c08d41 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:08.391294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 104/206] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:08.398722 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49079f7-3fff-443e-ab20-3a293ef20bed None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:08.530496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:08.554756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49079f7-3fff-443e-ab20-3a293ef20bed None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:08.593537 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe17e99-934f-4b1a-8227-5abb318e1f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:08.594152 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe17e99-934f-4b1a-8227-5abb318e1f70 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:08.594411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe17e99-934f-4b1a-8227-5abb318e1f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:08.594675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe17e99-934f-4b1a-8227-5abb318e1f70 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:08.595795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe17e99-934f-4b1a-8227-5abb318e1f70 None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:08.597305 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe17e99-934f-4b1a-8227-5abb318e1f70 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:08.599846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 103/207] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:08 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:08.605737 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aae9d5-55cb-479e-b635-50ec66c08d41 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:08.624025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:08.728714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aae9d5-55cb-479e-b635-50ec66c08d41 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:08.763055 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c0ea9-28a0-408c-9bf0-94a9c9ae4cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:08.763662 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c0ea9-28a0-408c-9bf0-94a9c9ae4cac None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:08.763940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c0ea9-28a0-408c-9bf0-94a9c9ae4cac None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:08.764236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c0ea9-28a0-408c-9bf0-94a9c9ae4cac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:08.764956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c0ea9-28a0-408c-9bf0-94a9c9ae4cac None admin] RBAC: Adding query filter params (role.id=8cf91ce8e7dd42aaa2708185cb13a6db, scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3, user.id=f152a86ae2354080a31a2daaeec2ef96) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:08.765256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c0ea9-28a0-408c-9bf0-94a9c9ae4cac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:08.766701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c0ea9-28a0-408c-9bf0-94a9c9ae4cac None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:08.772924 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 105/208] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:08 2022] GET /identity/v3/role_assignments?user.id=f152a86ae2354080a31a2daaeec2ef96&scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3&role.id=8cf91ce8e7dd42aaa2708185cb13a6db => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:09.557517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb13de6-7fa1-4a53-b117-eb779f1f41e8 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:09.557887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb13de6-7fa1-4a53-b117-eb779f1f41e8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:09.558180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb13de6-7fa1-4a53-b117-eb779f1f41e8 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:09.559905 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 104/209] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:09.565540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb730413-8d7a-4be1-b84d-feb4c5c8de57 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:09.565816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb730413-8d7a-4be1-b84d-feb4c5c8de57 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:09.566091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb730413-8d7a-4be1-b84d-feb4c5c8de57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:09.585428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-eb730413-8d7a-4be1-b84d-feb4c5c8de57 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:09.586610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-eb730413-8d7a-4be1-b84d-feb4c5c8de57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:09.655843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511c916-daab-4f80-b4b8-0d00fb1922a1 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:09.656395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511c916-daab-4f80-b4b8-0d00fb1922a1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:09.656395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511c916-daab-4f80-b4b8-0d00fb1922a1 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:09.657268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 106/210] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:09.657961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 105/211] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:09.663446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017d341-f7e8-448c-82e5-13f68413b9b9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:09.663732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017d341-f7e8-448c-82e5-13f68413b9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:09.664040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017d341-f7e8-448c-82e5-13f68413b9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:09.670934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:09.682705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a017d341-f7e8-448c-82e5-13f68413b9b9 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:09.683876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a017d341-f7e8-448c-82e5-13f68413b9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:09.756148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 107/212] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:09.939623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511c916-daab-4f80-b4b8-0d00fb1922a1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:10.040501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017d341-f7e8-448c-82e5-13f68413b9b9 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:10.053025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511c916-daab-4f80-b4b8-0d00fb1922a1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:10.096206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70150aac-f036-49c2-925a-d9d534fc33a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:10.096984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70150aac-f036-49c2-925a-d9d534fc33a9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:10.097280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70150aac-f036-49c2-925a-d9d534fc33a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:10.098158 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70150aac-f036-49c2-925a-d9d534fc33a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:10.113695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70150aac-f036-49c2-925a-d9d534fc33a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-70150aac-f036-49c2-925a-d9d534fc33a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:10.119138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:10.120041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:10.120844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:10.120844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:10.120844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:10.120844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:10.120844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:10.120844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:10.120844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:10.120844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:10.120844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:10.120844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:10.120844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:10.120844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:10.128109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 106/213] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:10.129986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70150aac-f036-49c2-925a-d9d534fc33a9 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:10.144029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017d341-f7e8-448c-82e5-13f68413b9b9 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:10.181827 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:10.182763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daac130-1483-4177-95fa-f5a926cef8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:10.183493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daac130-1483-4177-95fa-f5a926cef8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:10.183872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daac130-1483-4177-95fa-f5a926cef8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:10.184255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daac130-1483-4177-95fa-f5a926cef8e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:10.185455 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daac130-1483-4177-95fa-f5a926cef8e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:10.186990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daac130-1483-4177-95fa-f5a926cef8e3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:10.192106 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 108/214] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:10.196856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daac130-1483-4177-95fa-f5a926cef8e3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:10.199917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:10.245771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70150aac-f036-49c2-925a-d9d534fc33a9 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:10.283876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159a5ef-6450-4d76-8377-05d3ce565f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:10.284758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159a5ef-6450-4d76-8377-05d3ce565f7d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:10.285400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159a5ef-6450-4d76-8377-05d3ce565f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:10.285970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159a5ef-6450-4d76-8377-05d3ce565f7d None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:10.286899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159a5ef-6450-4d76-8377-05d3ce565f7d None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:10.287404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159a5ef-6450-4d76-8377-05d3ce565f7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:10.290157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159a5ef-6450-4d76-8377-05d3ce565f7d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:10.304364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 107/215] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:10.306083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159a5ef-6450-4d76-8377-05d3ce565f7d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:10.321016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daac130-1483-4177-95fa-f5a926cef8e3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:10.358858 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea4b13-128f-4559-8cd4-d695775e818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:10.359820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea4b13-128f-4559-8cd4-d695775e818d None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:10.360349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea4b13-128f-4559-8cd4-d695775e818d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:10.360824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea4b13-128f-4559-8cd4-d695775e818d None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:10.361709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea4b13-128f-4559-8cd4-d695775e818d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:10.364127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea4b13-128f-4559-8cd4-d695775e818d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:10.396645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:10.404688 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 109/216] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:16:10 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:16:10.435098 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159a5ef-6450-4d76-8377-05d3ce565f7d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:10.476158 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a79a4-8410-4bb1-a07b-e1111c94f73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:10.476734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a79a4-8410-4bb1-a07b-e1111c94f73b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:10.477017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a79a4-8410-4bb1-a07b-e1111c94f73b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:10.477290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a79a4-8410-4bb1-a07b-e1111c94f73b None admin] PATH_INFO: `/v3/users/528a8cb702984cecb2ea3a8da3c195cb` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:10.495542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a79a4-8410-4bb1-a07b-e1111c94f73b None admin] RBAC: Authorizing `identity:get_user(user_id=528a8cb702984cecb2ea3a8da3c195cb)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:10.497658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a79a4-8410-4bb1-a07b-e1111c94f73b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:10.499171 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef57bd-b69c-4cc7-92e3-a392c9a83826 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:10.499462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef57bd-b69c-4cc7-92e3-a392c9a83826 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:10.499739 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef57bd-b69c-4cc7-92e3-a392c9a83826 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:10.499868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 108/217] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:10 2022] GET /identity/v3/users/528a8cb702984cecb2ea3a8da3c195cb => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:10.501431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 110/218] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:16:10.506703 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a79a4-8410-4bb1-a07b-e1111c94f73b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:10.507265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fe5bd-aa17-4d80-b3e9-5a2c2897c22c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:10.507556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fe5bd-aa17-4d80-b3e9-5a2c2897c22c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:10.507899 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fe5bd-aa17-4d80-b3e9-5a2c2897c22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:10.527295 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-393fe5bd-aa17-4d80-b3e9-5a2c2897c22c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:10.528707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-393fe5bd-aa17-4d80-b3e9-5a2c2897c22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:10.600035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:10.622838 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 111/219] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:10.648766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a79a4-8410-4bb1-a07b-e1111c94f73b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:10.682724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:10.688216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efc1c9-a2ba-44be-a3f8-a8b9a20a6b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:10.689223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efc1c9-a2ba-44be-a3f8-a8b9a20a6b89 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:10.689714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efc1c9-a2ba-44be-a3f8-a8b9a20a6b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:10.690356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efc1c9-a2ba-44be-a3f8-a8b9a20a6b89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:10.739620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17eb93-52da-470d-8351-9744a7365cf7 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:10.739897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17eb93-52da-470d-8351-9744a7365cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:10.740125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17eb93-52da-470d-8351-9744a7365cf7 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:10.741472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 112/220] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:16:10.746684 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a02c53-97e6-4467-bf7b-276d6d4bb414 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:10.746965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a02c53-97e6-4467-bf7b-276d6d4bb414 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:10.747238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a02c53-97e6-4467-bf7b-276d6d4bb414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:10.769490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d5a02c53-97e6-4467-bf7b-276d6d4bb414 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:10.770682 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d5a02c53-97e6-4467-bf7b-276d6d4bb414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:10.773840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59efc1c9-a2ba-44be-a3f8-a8b9a20a6b89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:10.775336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59efc1c9-a2ba-44be-a3f8-a8b9a20a6b89 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:10.864400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 113/221] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:10.906852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 109/222] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:10.913092 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a02c53-97e6-4467-bf7b-276d6d4bb414 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:10.934470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efc1c9-a2ba-44be-a3f8-a8b9a20a6b89 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:10.976045 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:11.062129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a02c53-97e6-4467-bf7b-276d6d4bb414 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:11.087534 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efc1c9-a2ba-44be-a3f8-a8b9a20a6b89 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:11.097234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:11.129477 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12617182-cec4-4d6a-b687-e9fa2ea890c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:11.130318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12617182-cec4-4d6a-b687-e9fa2ea890c3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:11.130595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12617182-cec4-4d6a-b687-e9fa2ea890c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:11.130863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12617182-cec4-4d6a-b687-e9fa2ea890c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:11.131968 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fac6b5-bfa0-4ef8-886f-ce7cf8b1a630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:11.132614 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fac6b5-bfa0-4ef8-886f-ce7cf8b1a630 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:11.132885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fac6b5-bfa0-4ef8-886f-ce7cf8b1a630 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:11.133150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fac6b5-bfa0-4ef8-886f-ce7cf8b1a630 None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:11.135430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fac6b5-bfa0-4ef8-886f-ce7cf8b1a630 None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:11.136915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fac6b5-bfa0-4ef8-886f-ce7cf8b1a630 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:11.138753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 114/223] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:10 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:11.152971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12617182-cec4-4d6a-b687-e9fa2ea890c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-12617182-cec4-4d6a-b687-e9fa2ea890c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:11.157958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:11.158830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:11.160491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:11.160491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:11.160491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:11.160491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:11.160491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:11.160491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:11.160491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:11.160491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:11.160491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:11.160491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:11.160491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:11.160491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:11.160491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 110/224] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:16:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:16:11.163970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fac6b5-bfa0-4ef8-886f-ce7cf8b1a630 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:11.168065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12617182-cec4-4d6a-b687-e9fa2ea890c3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:11.288830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:11.289021 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fac6b5-bfa0-4ef8-886f-ce7cf8b1a630 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:11.300163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12617182-cec4-4d6a-b687-e9fa2ea890c3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:11.329921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ec8b7-679a-4177-bee7-2a3200000b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:11.330543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ec8b7-679a-4177-bee7-2a3200000b4b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:11.330895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ec8b7-679a-4177-bee7-2a3200000b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:11.331412 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ec8b7-679a-4177-bee7-2a3200000b4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:11.333255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ec8b7-679a-4177-bee7-2a3200000b4b None admin] RBAC: Adding query filter params (role.id=d9f6babaca2d494c9a04f0aca1271af8, scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17, user.id=528a8cb702984cecb2ea3a8da3c195cb) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:11.333847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ec8b7-679a-4177-bee7-2a3200000b4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:11.335331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ec8b7-679a-4177-bee7-2a3200000b4b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:11.337901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87528f93-2f79-4583-a309-5105b7df9edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:11.338769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87528f93-2f79-4583-a309-5105b7df9edf None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:11.339247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87528f93-2f79-4583-a309-5105b7df9edf None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:11.339715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87528f93-2f79-4583-a309-5105b7df9edf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:11.341419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 115/225] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:11 2022] GET /identity/v3/role_assignments?user.id=528a8cb702984cecb2ea3a8da3c195cb&scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17&role.id=d9f6babaca2d494c9a04f0aca1271af8 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:11.342341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ec8b7-679a-4177-bee7-2a3200000b4b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:11.352408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87528f93-2f79-4583-a309-5105b7df9edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-87528f93-2f79-4583-a309-5105b7df9edf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:11.357530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:11.358435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:11.359235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:11.359235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:11.359235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:11.359235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:11.359235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:11.359235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:11.359235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:11.359235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:11.359235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:11.359235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:11.359235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:11.359235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:11.359235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 111/226] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:16:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:11.365491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87528f93-2f79-4583-a309-5105b7df9edf None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:11.435353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:11.531740 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87528f93-2f79-4583-a309-5105b7df9edf None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:11.548208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ec8b7-679a-4177-bee7-2a3200000b4b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:11.575323 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd6675-95db-46c7-978e-20c114b84257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:11.576212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd6675-95db-46c7-978e-20c114b84257 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:11.576495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd6675-95db-46c7-978e-20c114b84257 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:11.576769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd6675-95db-46c7-978e-20c114b84257 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:11.577452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd6675-95db-46c7-978e-20c114b84257 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:11.577754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd6675-95db-46c7-978e-20c114b84257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:11.579185 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd6675-95db-46c7-978e-20c114b84257 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:11.590384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 112/227] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:11.596118 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd6675-95db-46c7-978e-20c114b84257 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:11.618698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:11.619673 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2affc16-cc74-4ca5-8235-55cff4677a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:11.620342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2affc16-cc74-4ca5-8235-55cff4677a8f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:11.620606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2affc16-cc74-4ca5-8235-55cff4677a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:11.620874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2affc16-cc74-4ca5-8235-55cff4677a8f None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:11.621767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2affc16-cc74-4ca5-8235-55cff4677a8f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:11.621874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2affc16-cc74-4ca5-8235-55cff4677a8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:11.623233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2affc16-cc74-4ca5-8235-55cff4677a8f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:11.634564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 116/228] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:11.641673 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2affc16-cc74-4ca5-8235-55cff4677a8f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:11.773729 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd6675-95db-46c7-978e-20c114b84257 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:11.807636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2affc16-cc74-4ca5-8235-55cff4677a8f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:11.823748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:11.850840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af14847-02ee-4611-9c70-08b7b5c640ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:11.851398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af14847-02ee-4611-9c70-08b7b5c640ad None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:11.851680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af14847-02ee-4611-9c70-08b7b5c640ad None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:11.851953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af14847-02ee-4611-9c70-08b7b5c640ad None admin] PATH_INFO: `/v3/users/8b5a7b7bd6b54fcf821970d4d9ada6a4` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:11.853136 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af14847-02ee-4611-9c70-08b7b5c640ad None admin] RBAC: Authorizing `identity:get_user(user_id=8b5a7b7bd6b54fcf821970d4d9ada6a4)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:11.854732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af14847-02ee-4611-9c70-08b7b5c640ad None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:11.856493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 117/229] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:11 2022] GET /identity/v3/users/8b5a7b7bd6b54fcf821970d4d9ada6a4 => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:11.862623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af14847-02ee-4611-9c70-08b7b5c640ad None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:11.884760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409ce57-1e36-4a88-8982-9329c84ec08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:11.885399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409ce57-1e36-4a88-8982-9329c84ec08d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:11.885676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409ce57-1e36-4a88-8982-9329c84ec08d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:11.885947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409ce57-1e36-4a88-8982-9329c84ec08d None admin] PATH_INFO: `/v3/users/8b5a7b7bd6b54fcf821970d4d9ada6a4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:11.887067 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409ce57-1e36-4a88-8982-9329c84ec08d None admin] RBAC: Authorizing `identity:get_user(user_id=8b5a7b7bd6b54fcf821970d4d9ada6a4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:11.896687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409ce57-1e36-4a88-8982-9329c84ec08d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:11.898347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 113/230] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:11 2022] GET /identity/v3/users/8b5a7b7bd6b54fcf821970d4d9ada6a4 => generated 283 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:11.904276 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409ce57-1e36-4a88-8982-9329c84ec08d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:12.002077 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:12.035971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409ce57-1e36-4a88-8982-9329c84ec08d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:12.053858 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af14847-02ee-4611-9c70-08b7b5c640ad None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:12.102703 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861d8cd-5031-4235-bd48-ce609d6c210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:12.103262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861d8cd-5031-4235-bd48-ce609d6c210d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:12.103541 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861d8cd-5031-4235-bd48-ce609d6c210d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:12.103812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861d8cd-5031-4235-bd48-ce609d6c210d None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:12.104959 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861d8cd-5031-4235-bd48-ce609d6c210d None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:12.106428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861d8cd-5031-4235-bd48-ce609d6c210d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:12.108366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 114/231] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:11 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:12.115790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861d8cd-5031-4235-bd48-ce609d6c210d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:12.124620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cae8e-59ca-45b1-97f4-2138feb4eaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:12.125326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cae8e-59ca-45b1-97f4-2138feb4eaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:12.125603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cae8e-59ca-45b1-97f4-2138feb4eaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:12.125870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cae8e-59ca-45b1-97f4-2138feb4eaf0 None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:12.127844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cae8e-59ca-45b1-97f4-2138feb4eaf0 None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:12.134377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cae8e-59ca-45b1-97f4-2138feb4eaf0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:12.136435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 118/232] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:11 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:12.142548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cae8e-59ca-45b1-97f4-2138feb4eaf0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:12.212341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:12.299134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861d8cd-5031-4235-bd48-ce609d6c210d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:12.310868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cae8e-59ca-45b1-97f4-2138feb4eaf0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:12.364235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28b609-7832-432d-8f9e-278799ab0130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:12.364817 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28b609-7832-432d-8f9e-278799ab0130 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:12.365095 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28b609-7832-432d-8f9e-278799ab0130 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:12.365583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28b609-7832-432d-8f9e-278799ab0130 None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17/users/8b5a7b7bd6b54fcf821970d4d9ada6a4/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:12.369876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28b609-7832-432d-8f9e-278799ab0130 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c6f2a4f66134019b9d71c524ab5cb17, user_id=8b5a7b7bd6b54fcf821970d4d9ada6a4, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:12.370501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e59f334-ffa0-430b-8e44-3b465d77946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:12.371214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59f334-ffa0-430b-8e44-3b465d77946b None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:12.371389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28b609-7832-432d-8f9e-278799ab0130 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:12.371483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59f334-ffa0-430b-8e44-3b465d77946b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:12.371757 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59f334-ffa0-430b-8e44-3b465d77946b None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17/users/8b5a7b7bd6b54fcf821970d4d9ada6a4/roles/6c6c7fa19ee342da915c90895e707620` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:12.373941 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59f334-ffa0-430b-8e44-3b465d77946b None admin] RBAC: Authorizing `identity:create_grant(project_id=7c6f2a4f66134019b9d71c524ab5cb17, user_id=8b5a7b7bd6b54fcf821970d4d9ada6a4, role_id=6c6c7fa19ee342da915c90895e707620)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:12.375435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e59f334-ffa0-430b-8e44-3b465d77946b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:12.392469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 119/233] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:16:12 2022] PUT /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17/users/8b5a7b7bd6b54fcf821970d4d9ada6a4/roles/6c6c7fa19ee342da915c90895e707620 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:16:12.400263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 115/234] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:16:12 2022] PUT /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17/users/8b5a7b7bd6b54fcf821970d4d9ada6a4/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:16:13.096042 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183843a-3166-40fe-a11f-90cdd14ad915 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:13.097051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183843a-3166-40fe-a11f-90cdd14ad915 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:13.098062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183843a-3166-40fe-a11f-90cdd14ad915 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:13.099858 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 120/235] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:13.105529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d95034-a560-440b-9852-bad1cd305d29 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:13.106413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d95034-a560-440b-9852-bad1cd305d29 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:13.106888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d95034-a560-440b-9852-bad1cd305d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:13.132677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-59d95034-a560-440b-9852-bad1cd305d29 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:13.133899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-59d95034-a560-440b-9852-bad1cd305d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:13.136038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:13.223876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 116/236] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:13.400189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f835b0-1ad8-4e12-9ddf-1da7e1d01c28 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:13.400814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f835b0-1ad8-4e12-9ddf-1da7e1d01c28 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:13.401291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f835b0-1ad8-4e12-9ddf-1da7e1d01c28 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:13.402825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 121/237] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:13.408153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd6fda-f410-451a-8c46-01af033e557b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:13.408444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd6fda-f410-451a-8c46-01af033e557b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:13.408723 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd6fda-f410-451a-8c46-01af033e557b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:13.437086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-acbd6fda-f410-451a-8c46-01af033e557b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:13.438304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-acbd6fda-f410-451a-8c46-01af033e557b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:13.530468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 117/238] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:13.565852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f835b0-1ad8-4e12-9ddf-1da7e1d01c28 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:13.717704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f835b0-1ad8-4e12-9ddf-1da7e1d01c28 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:13.727598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:13.767445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e5cdc-8140-433b-a69d-ce780c045322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:13.768112 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e5cdc-8140-433b-a69d-ce780c045322 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:13.768398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e5cdc-8140-433b-a69d-ce780c045322 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:13.768667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e5cdc-8140-433b-a69d-ce780c045322 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:13.781397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e5cdc-8140-433b-a69d-ce780c045322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-a17e5cdc-8140-433b-a69d-ce780c045322 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:13.787059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:13.787937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:13.789273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:13.789273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:13.789273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:13.789273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:13.789273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:13.789273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:13.789273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:13.789273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:13.789273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:13.789273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:13.789273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:13.789273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:13.789273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 122/239] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:13.799341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd6fda-f410-451a-8c46-01af033e557b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:13.889518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e5cdc-8140-433b-a69d-ce780c045322 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:13.913595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:13.954439 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd6fda-f410-451a-8c46-01af033e557b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:13.995635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed978446-0e25-408e-bb50-67939155beab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:13.996204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed978446-0e25-408e-bb50-67939155beab None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:13.996483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed978446-0e25-408e-bb50-67939155beab None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:13.996757 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed978446-0e25-408e-bb50-67939155beab None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:13.997431 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed978446-0e25-408e-bb50-67939155beab None admin] RBAC: Adding query filter params (name=reader) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:13.997734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed978446-0e25-408e-bb50-67939155beab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:13.999114 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed978446-0e25-408e-bb50-67939155beab None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:14.009769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 118/240] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:14.015476 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed978446-0e25-408e-bb50-67939155beab None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:14.112425 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e5cdc-8140-433b-a69d-ce780c045322 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:14.157142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed978446-0e25-408e-bb50-67939155beab None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:14.158535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cd29d-7f6e-4d9e-a104-d4abfe3c15e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:14.159189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cd29d-7f6e-4d9e-a104-d4abfe3c15e0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:14.159476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cd29d-7f6e-4d9e-a104-d4abfe3c15e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:14.160077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cd29d-7f6e-4d9e-a104-d4abfe3c15e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:14.162219 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cd29d-7f6e-4d9e-a104-d4abfe3c15e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:14.163698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cd29d-7f6e-4d9e-a104-d4abfe3c15e0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:14.165327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 123/241] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:14.168557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649359a-e079-456b-adbc-13094edf19a3 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:14.168557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649359a-e079-456b-adbc-13094edf19a3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:14.168557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649359a-e079-456b-adbc-13094edf19a3 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:14.168557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 124/242] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:14.173640 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649359a-e079-456b-adbc-13094edf19a3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:14.199301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:14.207623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a305cf-2e45-4ae9-b410-bc2a4db103e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:14.208552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a305cf-2e45-4ae9-b410-bc2a4db103e6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:14.209056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a305cf-2e45-4ae9-b410-bc2a4db103e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:14.209518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a305cf-2e45-4ae9-b410-bc2a4db103e6 None admin] PATH_INFO: `/v3/users/f04ba63f7a2d496caa03eacf0ab332b5` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:14.224191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a305cf-2e45-4ae9-b410-bc2a4db103e6 None admin] RBAC: Authorizing `identity:get_user(user_id=f04ba63f7a2d496caa03eacf0ab332b5)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:14.225208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:14.226544 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a305cf-2e45-4ae9-b410-bc2a4db103e6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:14.229047 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 119/243] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:14 2022] GET /identity/v3/users/f04ba63f7a2d496caa03eacf0ab332b5 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:14.230575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e09bcc-235e-4495-81ac-01d8cfd0213a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:14.231167 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e09bcc-235e-4495-81ac-01d8cfd0213a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:14.231638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e09bcc-235e-4495-81ac-01d8cfd0213a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:14.251674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-39e09bcc-235e-4495-81ac-01d8cfd0213a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:14.253193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-39e09bcc-235e-4495-81ac-01d8cfd0213a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:14.312489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649359a-e079-456b-adbc-13094edf19a3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:14.328762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 120/244] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:14.330290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e09bcc-235e-4495-81ac-01d8cfd0213a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:14.389344 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69db063-85b8-4920-a1dd-5f188d8a64b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:14.390379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69db063-85b8-4920-a1dd-5f188d8a64b6 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:14.390849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69db063-85b8-4920-a1dd-5f188d8a64b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:14.391307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69db063-85b8-4920-a1dd-5f188d8a64b6 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:14.392172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69db063-85b8-4920-a1dd-5f188d8a64b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:14.393965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69db063-85b8-4920-a1dd-5f188d8a64b6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:14.433446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 125/245] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:16:14 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:16:14.577263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e09bcc-235e-4495-81ac-01d8cfd0213a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:14.603030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:14.629031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebfb1c-dbcd-4d87-a513-741f20ac3053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:14.630080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebfb1c-dbcd-4d87-a513-741f20ac3053 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:14.630576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebfb1c-dbcd-4d87-a513-741f20ac3053 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:14.631040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebfb1c-dbcd-4d87-a513-741f20ac3053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:14.640445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69db063-85b8-4920-a1dd-5f188d8a64b6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:14.765246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebfb1c-dbcd-4d87-a513-741f20ac3053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:14.766763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebfb1c-dbcd-4d87-a513-741f20ac3053 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:14.770284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:14.782229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69db063-85b8-4920-a1dd-5f188d8a64b6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:14.842430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc4b69-29b7-424c-883e-0817379ffdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:14.842993 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc4b69-29b7-424c-883e-0817379ffdfb None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:14.843270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc4b69-29b7-424c-883e-0817379ffdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:14.843529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc4b69-29b7-424c-883e-0817379ffdfb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:14.856419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc4b69-29b7-424c-883e-0817379ffdfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-a9bc4b69-29b7-424c-883e-0817379ffdfb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:14.865895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:14.866837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:14.867623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:14.867623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:14.867623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:14.867623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:14.867623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:14.867623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:14.867623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:14.867623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:14.867623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:14.867623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:14.867623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:14.867623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:14.867623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 126/246] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:16:14.874555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc4b69-29b7-424c-883e-0817379ffdfb None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:14.908736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 121/247] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:14.914419 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebfb1c-dbcd-4d87-a513-741f20ac3053 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:14.935547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:14.988844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc4b69-29b7-424c-883e-0817379ffdfb None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:15.026628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146d1a3-9762-42a2-b1ef-674b8594e8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:15.027711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146d1a3-9762-42a2-b1ef-674b8594e8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:15.028226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146d1a3-9762-42a2-b1ef-674b8594e8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:15.028700 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146d1a3-9762-42a2-b1ef-674b8594e8a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:15.029567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146d1a3-9762-42a2-b1ef-674b8594e8a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:15.030068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146d1a3-9762-42a2-b1ef-674b8594e8a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:15.031686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146d1a3-9762-42a2-b1ef-674b8594e8a0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:15.040668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 127/248] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:15.046620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146d1a3-9762-42a2-b1ef-674b8594e8a0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:15.049073 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebfb1c-dbcd-4d87-a513-741f20ac3053 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:15.071866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:15.105320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc92c67-9fa6-4d9a-b71d-dd5ba8c70c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:15.106442 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc92c67-9fa6-4d9a-b71d-dd5ba8c70c99 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:15.107023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc92c67-9fa6-4d9a-b71d-dd5ba8c70c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:15.107496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc92c67-9fa6-4d9a-b71d-dd5ba8c70c99 None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:15.108895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc92c67-9fa6-4d9a-b71d-dd5ba8c70c99 None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:15.110839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc92c67-9fa6-4d9a-b71d-dd5ba8c70c99 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:15.113613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 122/249] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:14 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:15.119422 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc92c67-9fa6-4d9a-b71d-dd5ba8c70c99 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:15.173268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146d1a3-9762-42a2-b1ef-674b8594e8a0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:15.221994 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d7c22-43b1-406f-8f56-92292f613885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:15.223336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d7c22-43b1-406f-8f56-92292f613885 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:15.223608 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:15.224071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d7c22-43b1-406f-8f56-92292f613885 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:15.224071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d7c22-43b1-406f-8f56-92292f613885 None admin] PATH_INFO: `/v3/users/528a8cb702984cecb2ea3a8da3c195cb` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:15.225133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d7c22-43b1-406f-8f56-92292f613885 None admin] RBAC: Authorizing `identity:get_user(user_id=528a8cb702984cecb2ea3a8da3c195cb)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:15.226587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d7c22-43b1-406f-8f56-92292f613885 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:15.228451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 128/250] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:15 2022] GET /identity/v3/users/528a8cb702984cecb2ea3a8da3c195cb => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:15.229602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9bf6f-3dfd-408f-add1-a893bfed1d09 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:15.229772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9bf6f-3dfd-408f-add1-a893bfed1d09 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:15.230043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9bf6f-3dfd-408f-add1-a893bfed1d09 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:15.232049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 129/251] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:15.236622 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f267ae9-3308-4159-a610-d8246e4b86b4 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:15.236921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f267ae9-3308-4159-a610-d8246e4b86b4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:15.237202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f267ae9-3308-4159-a610-d8246e4b86b4 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:15.238896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 130/252] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:15.240103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45470a-8a1f-4709-8a70-c228256bdaa3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:15.240385 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45470a-8a1f-4709-8a70-c228256bdaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:15.240654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45470a-8a1f-4709-8a70-c228256bdaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:15.245759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc92c67-9fa6-4d9a-b71d-dd5ba8c70c99 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:15.261585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fa45470a-8a1f-4709-8a70-c228256bdaa3 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:15.262790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fa45470a-8a1f-4709-8a70-c228256bdaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:15.289375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0258b-417b-4446-9f84-4e896fb7f22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:15.289983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0258b-417b-4446-9f84-4e896fb7f22e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:15.290263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0258b-417b-4446-9f84-4e896fb7f22e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:15.290528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0258b-417b-4446-9f84-4e896fb7f22e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:15.291265 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0258b-417b-4446-9f84-4e896fb7f22e None admin] RBAC: Adding query filter params (role.id=8cf91ce8e7dd42aaa2708185cb13a6db, scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17, user.id=f04ba63f7a2d496caa03eacf0ab332b5) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:15.291573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0258b-417b-4446-9f84-4e896fb7f22e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:15.293068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0258b-417b-4446-9f84-4e896fb7f22e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:15.300486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 123/253] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:15 2022] GET /identity/v3/role_assignments?user.id=f04ba63f7a2d496caa03eacf0ab332b5&scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17&role.id=8cf91ce8e7dd42aaa2708185cb13a6db => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:15.301618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0258b-417b-4446-9f84-4e896fb7f22e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:15.361443 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 131/254] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:15.362758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d0ef4-ef80-4d25-aa02-b6b3a77a8721 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:15.363289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d0ef4-ef80-4d25-aa02-b6b3a77a8721 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:15.363781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d0ef4-ef80-4d25-aa02-b6b3a77a8721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:15.384741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ea1d0ef4-ef80-4d25-aa02-b6b3a77a8721 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:15.386353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ea1d0ef4-ef80-4d25-aa02-b6b3a77a8721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:15.395524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:15.465904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 132/255] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:15.525918 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0258b-417b-4446-9f84-4e896fb7f22e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:15.574217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b66c3a-e903-4166-a4a9-7199822802ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:15.574866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b66c3a-e903-4166-a4a9-7199822802ae None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:15.575145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b66c3a-e903-4166-a4a9-7199822802ae None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:15.575418 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b66c3a-e903-4166-a4a9-7199822802ae None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:15.576973 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b66c3a-e903-4166-a4a9-7199822802ae None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:15.578454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b66c3a-e903-4166-a4a9-7199822802ae None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:15.580122 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 124/256] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:15 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:15.586825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d0ef4-ef80-4d25-aa02-b6b3a77a8721 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:15.698172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d0ef4-ef80-4d25-aa02-b6b3a77a8721 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:15.733789 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aec141-4970-4068-8e03-a924ead1905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:15.734839 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aec141-4970-4068-8e03-a924ead1905f None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:15.735319 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aec141-4970-4068-8e03-a924ead1905f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:15.735845 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aec141-4970-4068-8e03-a924ead1905f None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17/users/528a8cb702984cecb2ea3a8da3c195cb/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:15.738524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aec141-4970-4068-8e03-a924ead1905f None admin] RBAC: Authorizing `identity:create_grant(project_id=7c6f2a4f66134019b9d71c524ab5cb17, user_id=528a8cb702984cecb2ea3a8da3c195cb, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:15.740207 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aec141-4970-4068-8e03-a924ead1905f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:15.761379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 133/257] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:16:15 2022] PUT /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17/users/528a8cb702984cecb2ea3a8da3c195cb/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:16:15.968401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b66c3a-e903-4166-a4a9-7199822802ae None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:15.971001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aec141-4970-4068-8e03-a924ead1905f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:15.979606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:15.997380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:16.122511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aec141-4970-4068-8e03-a924ead1905f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:16.162822 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abde3b-3305-4f13-97d9-176821590156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:16.163863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abde3b-3305-4f13-97d9-176821590156 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:16.164379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abde3b-3305-4f13-97d9-176821590156 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:16.164895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abde3b-3305-4f13-97d9-176821590156 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:16.178922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7abde3b-3305-4f13-97d9-176821590156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-f7abde3b-3305-4f13-97d9-176821590156 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:16.185678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:16.186556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:16.187372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:16.187372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:16.187372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:16.187372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:16.187372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:16.187372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:16.187372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:16.187372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:16.187372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:16.187372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:16.187372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:16.187372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:16.190093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 134/258] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:16:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:16:16.190411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b66c3a-e903-4166-a4a9-7199822802ae None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:16.196586 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abde3b-3305-4f13-97d9-176821590156 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:16.263524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e2244-400e-47d0-b2cd-832bd58e6e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:16.264196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e2244-400e-47d0-b2cd-832bd58e6e0f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:16.264386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e2244-400e-47d0-b2cd-832bd58e6e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:16.264666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e2244-400e-47d0-b2cd-832bd58e6e0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:16.271467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:16.287426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e2244-400e-47d0-b2cd-832bd58e6e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-3f5e2244-400e-47d0-b2cd-832bd58e6e0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:16.293108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:16.293995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:16.294822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:16.294822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:16.294822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:16.294822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:16.294822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:16.294822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:16.294822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:16.294822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:16.294822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:16.294822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:16.294822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:16.294822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:16.294822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 125/259] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:16:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:16.302840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e2244-400e-47d0-b2cd-832bd58e6e0f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:16.324090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abde3b-3305-4f13-97d9-176821590156 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:16.366384 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9dc47-e76a-4259-a2f1-962947c8d890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:16.367353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9dc47-e76a-4259-a2f1-962947c8d890 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:16.367840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9dc47-e76a-4259-a2f1-962947c8d890 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:16.368521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9dc47-e76a-4259-a2f1-962947c8d890 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:16.369534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9dc47-e76a-4259-a2f1-962947c8d890 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:16.370162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9dc47-e76a-4259-a2f1-962947c8d890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:16.371902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9dc47-e76a-4259-a2f1-962947c8d890 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:16.383086 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 135/260] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:16.389566 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9dc47-e76a-4259-a2f1-962947c8d890 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:16.473670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:16.497611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e2244-400e-47d0-b2cd-832bd58e6e0f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:16.521396 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9dc47-e76a-4259-a2f1-962947c8d890 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:16.544212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8509a-607e-4868-8612-2c555b069c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:16.545338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8509a-607e-4868-8612-2c555b069c60 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:16.545808 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8509a-607e-4868-8612-2c555b069c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:16.546296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8509a-607e-4868-8612-2c555b069c60 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:16.546404 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:16.547719 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8509a-607e-4868-8612-2c555b069c60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:16.548279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8509a-607e-4868-8612-2c555b069c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:16.550252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8509a-607e-4868-8612-2c555b069c60 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:16.560859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4581898-b309-4084-9068-d28f0593d4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:16.561584 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4581898-b309-4084-9068-d28f0593d4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:16.561951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4581898-b309-4084-9068-d28f0593d4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:16.562276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4581898-b309-4084-9068-d28f0593d4e5 None admin] PATH_INFO: `/v3/users/8b5a7b7bd6b54fcf821970d4d9ada6a4` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:16.563116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 126/261] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:16.563600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4581898-b309-4084-9068-d28f0593d4e5 None admin] RBAC: Authorizing `identity:get_user(user_id=8b5a7b7bd6b54fcf821970d4d9ada6a4)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:16.567786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4581898-b309-4084-9068-d28f0593d4e5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:16.567786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 136/262] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:16 2022] GET /identity/v3/users/8b5a7b7bd6b54fcf821970d4d9ada6a4 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:16.569164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8509a-607e-4868-8612-2c555b069c60 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:16.573102 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4581898-b309-4084-9068-d28f0593d4e5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:16.775293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8509a-607e-4868-8612-2c555b069c60 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:16.785492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4581898-b309-4084-9068-d28f0593d4e5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:16.842053 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee999a2-6638-48e3-bfb2-427d2610e12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:16.843034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee999a2-6638-48e3-bfb2-427d2610e12d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:16.844081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee999a2-6638-48e3-bfb2-427d2610e12d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:16.844703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee999a2-6638-48e3-bfb2-427d2610e12d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:16.852717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f02e8c-1ba5-4c78-bf9b-0e0ae308c6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:16.853676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f02e8c-1ba5-4c78-bf9b-0e0ae308c6f8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:16.855899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f02e8c-1ba5-4c78-bf9b-0e0ae308c6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:16.856430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f02e8c-1ba5-4c78-bf9b-0e0ae308c6f8 None admin] PATH_INFO: `/v3/users/8b5a7b7bd6b54fcf821970d4d9ada6a4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:16.861908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f02e8c-1ba5-4c78-bf9b-0e0ae308c6f8 None admin] RBAC: Authorizing `identity:get_user(user_id=8b5a7b7bd6b54fcf821970d4d9ada6a4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:16.863686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f02e8c-1ba5-4c78-bf9b-0e0ae308c6f8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:16.865526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 127/263] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:16 2022] GET /identity/v3/users/8b5a7b7bd6b54fcf821970d4d9ada6a4 => generated 283 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:16.871262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f02e8c-1ba5-4c78-bf9b-0e0ae308c6f8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:16.892988 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:16.965274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:16.993024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee999a2-6638-48e3-bfb2-427d2610e12d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:16.994498 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee999a2-6638-48e3-bfb2-427d2610e12d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:17.054455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f02e8c-1ba5-4c78-bf9b-0e0ae308c6f8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:17.121474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab11a3-a970-481a-9dfb-c133acfbf67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:17.122197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab11a3-a970-481a-9dfb-c133acfbf67a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:17.123020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab11a3-a970-481a-9dfb-c133acfbf67a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:17.123301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab11a3-a970-481a-9dfb-c133acfbf67a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:17.154946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 137/264] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:17.160523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee999a2-6638-48e3-bfb2-427d2610e12d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:17.261871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab11a3-a970-481a-9dfb-c133acfbf67a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:17.263455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab11a3-a970-481a-9dfb-c133acfbf67a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:17.282532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:17.292020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:17.310108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee999a2-6638-48e3-bfb2-427d2610e12d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:17.382584 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b6f7b-db54-4748-9ea8-e94e0d4485b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:17.383236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b6f7b-db54-4748-9ea8-e94e0d4485b4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:17.383519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b6f7b-db54-4748-9ea8-e94e0d4485b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:17.383791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b6f7b-db54-4748-9ea8-e94e0d4485b4 None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:17.384950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b6f7b-db54-4748-9ea8-e94e0d4485b4 None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:17.386421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b6f7b-db54-4748-9ea8-e94e0d4485b4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:17.400355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 138/265] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:17 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:17.401265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b6f7b-db54-4748-9ea8-e94e0d4485b4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:17.415633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 128/266] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:17.422820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab11a3-a970-481a-9dfb-c133acfbf67a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:17.584071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab11a3-a970-481a-9dfb-c133acfbf67a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:17.601457 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:17.636165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f46fe-71c9-4783-adb3-6c5e564dde0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:17.637160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f46fe-71c9-4783-adb3-6c5e564dde0e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:17.637650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f46fe-71c9-4783-adb3-6c5e564dde0e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:17.638126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f46fe-71c9-4783-adb3-6c5e564dde0e None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:17.641502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f46fe-71c9-4783-adb3-6c5e564dde0e None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:17.643239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f46fe-71c9-4783-adb3-6c5e564dde0e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:17.645400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 129/267] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:17 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:17.651953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f46fe-71c9-4783-adb3-6c5e564dde0e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:17.653790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b6f7b-db54-4748-9ea8-e94e0d4485b4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:17.724711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3c4b2-6ccd-4a4f-97d9-e33a6405b570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:17.725382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3c4b2-6ccd-4a4f-97d9-e33a6405b570 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:17.725662 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3c4b2-6ccd-4a4f-97d9-e33a6405b570 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:17.725980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3c4b2-6ccd-4a4f-97d9-e33a6405b570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:17.726721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3c4b2-6ccd-4a4f-97d9-e33a6405b570 None admin] RBAC: Adding query filter params (role.id=6c6c7fa19ee342da915c90895e707620, scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17, user.id=8b5a7b7bd6b54fcf821970d4d9ada6a4) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:17.727028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3c4b2-6ccd-4a4f-97d9-e33a6405b570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:17.728635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3c4b2-6ccd-4a4f-97d9-e33a6405b570 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:17.737514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 139/268] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:17 2022] GET /identity/v3/role_assignments?user.id=8b5a7b7bd6b54fcf821970d4d9ada6a4&scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17&role.id=6c6c7fa19ee342da915c90895e707620 => generated 608 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:17.835803 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f46fe-71c9-4783-adb3-6c5e564dde0e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:17.898161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f99d89-6c65-483a-ab4f-ea8a10f84c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:17.899159 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f99d89-6c65-483a-ab4f-ea8a10f84c06 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:17.899647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f99d89-6c65-483a-ab4f-ea8a10f84c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:17.900176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f99d89-6c65-483a-ab4f-ea8a10f84c06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:17.901203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f99d89-6c65-483a-ab4f-ea8a10f84c06 None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17, user.id=8b5a7b7bd6b54fcf821970d4d9ada6a4) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:17.901748 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f99d89-6c65-483a-ab4f-ea8a10f84c06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:17.903450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f99d89-6c65-483a-ab4f-ea8a10f84c06 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:17.913316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 130/269] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:17 2022] GET /identity/v3/role_assignments?user.id=8b5a7b7bd6b54fcf821970d4d9ada6a4&scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:18.103828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9befd-9367-4ff6-8196-2ac9cd04e196 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:18.104203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9befd-9367-4ff6-8196-2ac9cd04e196 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:18.104330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9befd-9367-4ff6-8196-2ac9cd04e196 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:18.105864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 140/270] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:18.111679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef1a4b-f2be-4fe5-b746-c0092e1cb81c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:18.112362 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef1a4b-f2be-4fe5-b746-c0092e1cb81c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:18.113431 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef1a4b-f2be-4fe5-b746-c0092e1cb81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:18.134161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-05ef1a4b-f2be-4fe5-b746-c0092e1cb81c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:18.135720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-05ef1a4b-f2be-4fe5-b746-c0092e1cb81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:18.179720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:18.220800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 131/271] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:18.291920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365baa04-c104-4dd9-8362-abde38d55f80 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:18.292238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365baa04-c104-4dd9-8362-abde38d55f80 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:18.292774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365baa04-c104-4dd9-8362-abde38d55f80 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:18.294096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 141/272] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:18.299580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03bd87-8b09-49ff-881a-e752a5171d9f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:18.300028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03bd87-8b09-49ff-881a-e752a5171d9f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:18.300226 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03bd87-8b09-49ff-881a-e752a5171d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:18.320916 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ce03bd87-8b09-49ff-881a-e752a5171d9f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:18.322353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ce03bd87-8b09-49ff-881a-e752a5171d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:18.404196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 132/273] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:18.548487 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365baa04-c104-4dd9-8362-abde38d55f80 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:18.578120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:18.666768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c98b8-6fa7-4e4c-b245-92c9389fffde None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:18.667402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c98b8-6fa7-4e4c-b245-92c9389fffde None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:18.667884 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c98b8-6fa7-4e4c-b245-92c9389fffde None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:18.669766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 133/274] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:16:18.674969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a8c58-603a-4ea8-9f20-6a56998cf439 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:18.676112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a8c58-603a-4ea8-9f20-6a56998cf439 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:18.676449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a8c58-603a-4ea8-9f20-6a56998cf439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:18.677515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365baa04-c104-4dd9-8362-abde38d55f80 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:18.697240 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-324a8c58-603a-4ea8-9f20-6a56998cf439 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:18.698543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-324a8c58-603a-4ea8-9f20-6a56998cf439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:18.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209cde6-a14f-48d6-89b6-da41680ad1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:18.724741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209cde6-a14f-48d6-89b6-da41680ad1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:18.724741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209cde6-a14f-48d6-89b6-da41680ad1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:18.724741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209cde6-a14f-48d6-89b6-da41680ad1b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:18.734698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209cde6-a14f-48d6-89b6-da41680ad1b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-4209cde6-a14f-48d6-89b6-da41680ad1b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:18.740323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:18.741203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:18.742004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:18.742004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:18.742004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:18.742004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:18.742004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:18.742004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:18.742004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:18.742004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:18.742004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:18.742004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:18.742004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:18.742004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:18.742004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 142/275] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:18.743208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209cde6-a14f-48d6-89b6-da41680ad1b9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:18.788780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 134/276] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:18.790038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a8c58-603a-4ea8-9f20-6a56998cf439 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:18.825619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:18.851451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:18.868879 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209cde6-a14f-48d6-89b6-da41680ad1b9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:18.906783 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e335fa-8a79-4fcb-aac8-20eef2a7a441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:18.907426 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e335fa-8a79-4fcb-aac8-20eef2a7a441 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:18.907706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e335fa-8a79-4fcb-aac8-20eef2a7a441 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:18.907989 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e335fa-8a79-4fcb-aac8-20eef2a7a441 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:18.909145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e335fa-8a79-4fcb-aac8-20eef2a7a441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:18.910456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a8c58-603a-4ea8-9f20-6a56998cf439 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:18.910668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e335fa-8a79-4fcb-aac8-20eef2a7a441 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:18.916083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 143/277] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:18.918212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e335fa-8a79-4fcb-aac8-20eef2a7a441 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:18.948830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900e20f-d00c-4756-9fd2-e244191727d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:18.949441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900e20f-d00c-4756-9fd2-e244191727d3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:18.949718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900e20f-d00c-4756-9fd2-e244191727d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:18.949992 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900e20f-d00c-4756-9fd2-e244191727d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:18.950658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900e20f-d00c-4756-9fd2-e244191727d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:18.950962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900e20f-d00c-4756-9fd2-e244191727d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:18.952379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900e20f-d00c-4756-9fd2-e244191727d3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:18.961510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 135/278] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:18.962510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500823b7-d627-4162-a58a-ba3be909ab69 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:18.962795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500823b7-d627-4162-a58a-ba3be909ab69 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:18.963071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500823b7-d627-4162-a58a-ba3be909ab69 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:18.968052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 136/279] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:18.969208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500823b7-d627-4162-a58a-ba3be909ab69 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:19.028425 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e335fa-8a79-4fcb-aac8-20eef2a7a441 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:19.067191 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf317b0-6dd0-4637-9bd1-d3925d6af7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:19.067619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:19.068185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf317b0-6dd0-4637-9bd1-d3925d6af7d4 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:19.068472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf317b0-6dd0-4637-9bd1-d3925d6af7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:19.068901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf317b0-6dd0-4637-9bd1-d3925d6af7d4 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:19.069556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf317b0-6dd0-4637-9bd1-d3925d6af7d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:19.071012 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf317b0-6dd0-4637-9bd1-d3925d6af7d4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:19.077760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500823b7-d627-4162-a58a-ba3be909ab69 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:19.092215 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 144/280] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:16:18 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:16:19.094450 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f0b11-5ef4-422a-81df-1414c6b9b004 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:19.094926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f0b11-5ef4-422a-81df-1414c6b9b004 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:19.095199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f0b11-5ef4-422a-81df-1414c6b9b004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:19.114975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4e1f0b11-5ef4-422a-81df-1414c6b9b004 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:19.116252 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4e1f0b11-5ef4-422a-81df-1414c6b9b004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:19.118558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:19.131050 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a16dac-97e8-4a93-bea0-e58b5bed8e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:19.131955 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a16dac-97e8-4a93-bea0-e58b5bed8e57 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:19.132274 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a16dac-97e8-4a93-bea0-e58b5bed8e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:19.132542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a16dac-97e8-4a93-bea0-e58b5bed8e57 None admin] PATH_INFO: `/v3/users/a825a3c7a1da417e9db6805e133b8f01` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:19.148153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a16dac-97e8-4a93-bea0-e58b5bed8e57 None admin] RBAC: Authorizing `identity:get_user(user_id=a825a3c7a1da417e9db6805e133b8f01)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:19.149662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a16dac-97e8-4a93-bea0-e58b5bed8e57 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:19.151617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 137/281] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:18 2022] GET /identity/v3/users/a825a3c7a1da417e9db6805e133b8f01 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:19.152281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a16dac-97e8-4a93-bea0-e58b5bed8e57 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:19.206436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 145/282] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:19.207955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f0b11-5ef4-422a-81df-1414c6b9b004 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:19.286871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a16dac-97e8-4a93-bea0-e58b5bed8e57 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:19.344110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1fba9-264c-48c1-8c08-c458557fb307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:19.346553 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1fba9-264c-48c1-8c08-c458557fb307 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:19.346832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1fba9-264c-48c1-8c08-c458557fb307 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:19.347093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1fba9-264c-48c1-8c08-c458557fb307 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:19.360119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f0b11-5ef4-422a-81df-1414c6b9b004 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:19.362054 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1fba9-264c-48c1-8c08-c458557fb307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-74a1fba9-264c-48c1-8c08-c458557fb307 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:19.367680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:19.368618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:19.369846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:19.369846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:19.369846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:19.369846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:19.369846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:19.369846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:19.369846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:19.369846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:19.369846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:19.369846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:19.369846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:19.369846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:19.369846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 138/283] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:19.375554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1fba9-264c-48c1-8c08-c458557fb307 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:19.390231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:19.414011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4f86c-b0b0-44c4-9970-1e66feedd3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:19.414628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4f86c-b0b0-44c4-9970-1e66feedd3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:19.414907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4f86c-b0b0-44c4-9970-1e66feedd3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:19.415182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4f86c-b0b0-44c4-9970-1e66feedd3c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:19.415914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4f86c-b0b0-44c4-9970-1e66feedd3c9 None admin] RBAC: Adding query filter params (role.id=d9f6babaca2d494c9a04f0aca1271af8, user.id=a825a3c7a1da417e9db6805e133b8f01, scope.system=all) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:19.416344 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4f86c-b0b0-44c4-9970-1e66feedd3c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:19.417785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4f86c-b0b0-44c4-9970-1e66feedd3c9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:19.425339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 146/284] 10.222.0.33 () {56 vars in 1308 bytes} [Sun Sep 11 05:16:19 2022] GET /identity/v3/role_assignments?user.id=a825a3c7a1da417e9db6805e133b8f01&scope.system=all&role.id=d9f6babaca2d494c9a04f0aca1271af8 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:19.505594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1fba9-264c-48c1-8c08-c458557fb307 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:19.507960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4f86c-b0b0-44c4-9970-1e66feedd3c9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:19.512498 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:19.551854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f17f60-31af-4aa8-b04e-245b1dd8e975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:19.552537 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f17f60-31af-4aa8-b04e-245b1dd8e975 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:19.552815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f17f60-31af-4aa8-b04e-245b1dd8e975 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:19.553083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f17f60-31af-4aa8-b04e-245b1dd8e975 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:19.553786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f17f60-31af-4aa8-b04e-245b1dd8e975 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:19.554085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f17f60-31af-4aa8-b04e-245b1dd8e975 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:19.555489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f17f60-31af-4aa8-b04e-245b1dd8e975 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:19.564874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 139/285] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:19.573483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f17f60-31af-4aa8-b04e-245b1dd8e975 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:19.646925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4f86c-b0b0-44c4-9970-1e66feedd3c9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:19.671589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:19.700700 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c591be-d060-4884-bcee-0ee122e714cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:19.701577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c591be-d060-4884-bcee-0ee122e714cd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:19.702021 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c591be-d060-4884-bcee-0ee122e714cd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:19.702696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c591be-d060-4884-bcee-0ee122e714cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:19.709391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f17f60-31af-4aa8-b04e-245b1dd8e975 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:19.716316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c591be-d060-4884-bcee-0ee122e714cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-b9c591be-d060-4884-bcee-0ee122e714cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:19.724140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:19.725243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:19.726211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:19.726211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:19.726211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:19.726211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:19.726211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:19.726211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:19.726211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:19.726211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:19.726211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:19.726211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:19.726211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:19.726211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:19.728542 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 147/286] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:19.734431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c591be-d060-4884-bcee-0ee122e714cd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:19.779965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3200c9-89bd-458b-bd50-9a234cd68b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:19.780859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3200c9-89bd-458b-bd50-9a234cd68b6d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:19.781304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3200c9-89bd-458b-bd50-9a234cd68b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:19.781715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3200c9-89bd-458b-bd50-9a234cd68b6d None admin] PATH_INFO: `/v3/users/f04ba63f7a2d496caa03eacf0ab332b5` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:19.783590 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3200c9-89bd-458b-bd50-9a234cd68b6d None admin] RBAC: Authorizing `identity:get_user(user_id=f04ba63f7a2d496caa03eacf0ab332b5)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:19.785829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3200c9-89bd-458b-bd50-9a234cd68b6d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:19.787827 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 140/287] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:19 2022] GET /identity/v3/users/f04ba63f7a2d496caa03eacf0ab332b5 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:19.795184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3200c9-89bd-458b-bd50-9a234cd68b6d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:19.860357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:19.933399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c591be-d060-4884-bcee-0ee122e714cd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:19.990430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3200c9-89bd-458b-bd50-9a234cd68b6d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:19.997059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad133090-34e9-41c8-9678-e046a7a90940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:19.997672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad133090-34e9-41c8-9678-e046a7a90940 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:19.997944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad133090-34e9-41c8-9678-e046a7a90940 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:19.998210 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad133090-34e9-41c8-9678-e046a7a90940 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:19.998876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad133090-34e9-41c8-9678-e046a7a90940 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:19.999170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad133090-34e9-41c8-9678-e046a7a90940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:20.000589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad133090-34e9-41c8-9678-e046a7a90940 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:20.011572 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 148/288] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:20.021049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad133090-34e9-41c8-9678-e046a7a90940 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:20.061804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb253e71-ce1b-4725-b5d8-aa0a78cc5a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:20.063026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb253e71-ce1b-4725-b5d8-aa0a78cc5a36 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:20.065803 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb253e71-ce1b-4725-b5d8-aa0a78cc5a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:20.066406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb253e71-ce1b-4725-b5d8-aa0a78cc5a36 None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:20.068624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb253e71-ce1b-4725-b5d8-aa0a78cc5a36 None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:20.070532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb253e71-ce1b-4725-b5d8-aa0a78cc5a36 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:20.073537 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 141/289] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:19 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:20.080637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb253e71-ce1b-4725-b5d8-aa0a78cc5a36 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:20.104784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:20.229339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad133090-34e9-41c8-9678-e046a7a90940 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:20.265201 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb253e71-ce1b-4725-b5d8-aa0a78cc5a36 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:20.292199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:20.314897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a90df-6755-49c6-9836-3c58e7abe3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:20.317245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a90df-6755-49c6-9836-3c58e7abe3c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:20.317945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a90df-6755-49c6-9836-3c58e7abe3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:20.318857 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a90df-6755-49c6-9836-3c58e7abe3c3 None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17/users/f04ba63f7a2d496caa03eacf0ab332b5/roles/8cf91ce8e7dd42aaa2708185cb13a6db` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:20.321723 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470fda4-faf5-4eac-be7a-6732b12200f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:20.322614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a90df-6755-49c6-9836-3c58e7abe3c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c6f2a4f66134019b9d71c524ab5cb17, user_id=f04ba63f7a2d496caa03eacf0ab332b5, role_id=8cf91ce8e7dd42aaa2708185cb13a6db)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:20.323511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470fda4-faf5-4eac-be7a-6732b12200f1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:20.324566 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470fda4-faf5-4eac-be7a-6732b12200f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:20.325764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470fda4-faf5-4eac-be7a-6732b12200f1 None admin] PATH_INFO: `/v3/users/528a8cb702984cecb2ea3a8da3c195cb` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:20.326962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a90df-6755-49c6-9836-3c58e7abe3c3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:20.327834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470fda4-faf5-4eac-be7a-6732b12200f1 None admin] RBAC: Authorizing `identity:get_user(user_id=528a8cb702984cecb2ea3a8da3c195cb)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:20.332097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470fda4-faf5-4eac-be7a-6732b12200f1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:20.337135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 149/290] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:20 2022] GET /identity/v3/users/528a8cb702984cecb2ea3a8da3c195cb => generated 297 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:20.343200 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470fda4-faf5-4eac-be7a-6732b12200f1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:20.354217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 142/291] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:16:20 2022] PUT /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17/users/f04ba63f7a2d496caa03eacf0ab332b5/roles/8cf91ce8e7dd42aaa2708185cb13a6db => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:16:20.486185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470fda4-faf5-4eac-be7a-6732b12200f1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:20.539204 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e10f0-a2e2-4507-9b8b-1674bed91dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:20.539840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e10f0-a2e2-4507-9b8b-1674bed91dbd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:20.540152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e10f0-a2e2-4507-9b8b-1674bed91dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:20.540426 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e10f0-a2e2-4507-9b8b-1674bed91dbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:20.608844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:20.619319 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e10f0-a2e2-4507-9b8b-1674bed91dbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:20.622265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e10f0-a2e2-4507-9b8b-1674bed91dbd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:20.726696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 150/292] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:20.732337 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a90df-6755-49c6-9836-3c58e7abe3c3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:20.861274 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:20.914935 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a90df-6755-49c6-9836-3c58e7abe3c3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:20.960304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c14a20-aa54-496a-9a71-5ae11b695e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:20.961328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c14a20-aa54-496a-9a71-5ae11b695e9e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:20.961815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c14a20-aa54-496a-9a71-5ae11b695e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:20.962319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c14a20-aa54-496a-9a71-5ae11b695e9e None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:20.963639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c14a20-aa54-496a-9a71-5ae11b695e9e None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:20.965308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c14a20-aa54-496a-9a71-5ae11b695e9e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:20.967486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 143/293] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:20 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:20.976024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e10f0-a2e2-4507-9b8b-1674bed91dbd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:21.107348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:21.123016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e10f0-a2e2-4507-9b8b-1674bed91dbd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:21.167397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab7088-9db3-4f00-a87b-29eda8aa035c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:21.168026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab7088-9db3-4f00-a87b-29eda8aa035c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:21.168315 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab7088-9db3-4f00-a87b-29eda8aa035c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:21.168588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab7088-9db3-4f00-a87b-29eda8aa035c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:21.169321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab7088-9db3-4f00-a87b-29eda8aa035c None admin] RBAC: Adding query filter params (role.id=d9f6babaca2d494c9a04f0aca1271af8, scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17, user.id=528a8cb702984cecb2ea3a8da3c195cb) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:21.169629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab7088-9db3-4f00-a87b-29eda8aa035c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:21.171083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab7088-9db3-4f00-a87b-29eda8aa035c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:21.177683 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 151/294] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:20 2022] GET /identity/v3/role_assignments?user.id=528a8cb702984cecb2ea3a8da3c195cb&scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17&role.id=d9f6babaca2d494c9a04f0aca1271af8 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:21.718748 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01e7ba-5ee6-474e-9fe7-43ce82f47a0d None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:21.719254 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01e7ba-5ee6-474e-9fe7-43ce82f47a0d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:21.719329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01e7ba-5ee6-474e-9fe7-43ce82f47a0d None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:21.720945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 144/295] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:21.727097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbd5b6-cd93-4a1a-ad40-6dab063a0a75 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:21.728027 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbd5b6-cd93-4a1a-ad40-6dab063a0a75 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:21.728516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbd5b6-cd93-4a1a-ad40-6dab063a0a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:21.748376 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3dbbd5b6-cd93-4a1a-ad40-6dab063a0a75 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:21.749902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3dbbd5b6-cd93-4a1a-ad40-6dab063a0a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:21.828307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 152/296] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:22.047394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79acb4d-ad35-46df-96c5-f565b5876b25 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:22.047749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79acb4d-ad35-46df-96c5-f565b5876b25 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:22.047923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79acb4d-ad35-46df-96c5-f565b5876b25 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:22.049502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 145/297] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:22.054825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632c407-552a-487d-8d7c-f8f8c85414bb None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:22.055358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632c407-552a-487d-8d7c-f8f8c85414bb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:22.055830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632c407-552a-487d-8d7c-f8f8c85414bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:22.074709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2632c407-552a-487d-8d7c-f8f8c85414bb None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:22.076235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2632c407-552a-487d-8d7c-f8f8c85414bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:22.105055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79acb4d-ad35-46df-96c5-f565b5876b25 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:22.156022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 153/298] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:22.156022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43bb36-6609-4fab-9ee7-70d9c9d0a4b4 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:22.156022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43bb36-6609-4fab-9ee7-70d9c9d0a4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:22.156022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43bb36-6609-4fab-9ee7-70d9c9d0a4b4 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:22.156022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 154/299] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:22.161101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73939770-9a14-4e57-822b-00bdc06ee6f3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:22.161390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73939770-9a14-4e57-822b-00bdc06ee6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:22.161946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73939770-9a14-4e57-822b-00bdc06ee6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:22.181158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-73939770-9a14-4e57-822b-00bdc06ee6f3 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:22.182341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-73939770-9a14-4e57-822b-00bdc06ee6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:22.196031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:22.220166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79acb4d-ad35-46df-96c5-f565b5876b25 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:22.259054 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d8864-b7b0-41fd-b4b3-d9a7fd589aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:22.259688 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d8864-b7b0-41fd-b4b3-d9a7fd589aec None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:22.259971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d8864-b7b0-41fd-b4b3-d9a7fd589aec None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:22.260279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d8864-b7b0-41fd-b4b3-d9a7fd589aec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:22.262564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 155/300] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:22.277360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d8864-b7b0-41fd-b4b3-d9a7fd589aec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-010d8864-b7b0-41fd-b4b3-d9a7fd589aec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:22.282763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:22.283639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:22.284481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:22.284481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:22.284481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:22.284481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:22.284481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:22.284481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:22.284481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:22.284481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:22.284481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:22.284481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:22.284481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:22.284481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:22.284481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 146/301] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:22.286408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:22.293374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73939770-9a14-4e57-822b-00bdc06ee6f3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:22.404649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73939770-9a14-4e57-822b-00bdc06ee6f3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:22.436614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d8864-b7b0-41fd-b4b3-d9a7fd589aec None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:22.439822 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf9da2-3819-4079-a6cf-cf584ac76c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:22.440837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf9da2-3819-4079-a6cf-cf584ac76c39 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:22.441335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf9da2-3819-4079-a6cf-cf584ac76c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:22.441811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf9da2-3819-4079-a6cf-cf584ac76c39 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:22.442681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf9da2-3819-4079-a6cf-cf584ac76c39 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:22.443181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf9da2-3819-4079-a6cf-cf584ac76c39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:22.444814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf9da2-3819-4079-a6cf-cf584ac76c39 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:22.454065 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 156/302] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:22.461252 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf9da2-3819-4079-a6cf-cf584ac76c39 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:22.474037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:22.566478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d8864-b7b0-41fd-b4b3-d9a7fd589aec None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:22.572476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf9da2-3819-4079-a6cf-cf584ac76c39 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:22.604119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9edb24-1b09-4899-8811-034d230cca59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:22.604749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9edb24-1b09-4899-8811-034d230cca59 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:22.605026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9edb24-1b09-4899-8811-034d230cca59 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:22.605295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9edb24-1b09-4899-8811-034d230cca59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:22.606482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9edb24-1b09-4899-8811-034d230cca59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:22.607941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9edb24-1b09-4899-8811-034d230cca59 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:22.609536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 147/303] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:22.610292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9edb24-1b09-4899-8811-034d230cca59 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:22.613630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fed69c-3162-4fc5-a014-4df7f16adf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:22.614501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fed69c-3162-4fc5-a014-4df7f16adf17 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:22.615280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fed69c-3162-4fc5-a014-4df7f16adf17 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:22.615914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fed69c-3162-4fc5-a014-4df7f16adf17 None admin] PATH_INFO: `/v3/users/c8ac14a93b1642429beb53a33ec96ae2` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:22.633866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fed69c-3162-4fc5-a014-4df7f16adf17 None admin] RBAC: Authorizing `identity:get_user(user_id=c8ac14a93b1642429beb53a33ec96ae2)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:22.635334 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fed69c-3162-4fc5-a014-4df7f16adf17 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:22.641579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 157/304] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:22 2022] GET /identity/v3/users/c8ac14a93b1642429beb53a33ec96ae2 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:22.641579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fed69c-3162-4fc5-a014-4df7f16adf17 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:22.686047 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:22.711449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:22.732609 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9edb24-1b09-4899-8811-034d230cca59 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:22.749523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fed69c-3162-4fc5-a014-4df7f16adf17 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:22.768883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30327a-7625-4b6e-9ef6-64389e5c04c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:22.769555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30327a-7625-4b6e-9ef6-64389e5c04c2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:22.769859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30327a-7625-4b6e-9ef6-64389e5c04c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:22.770132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30327a-7625-4b6e-9ef6-64389e5c04c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:22.780678 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30327a-7625-4b6e-9ef6-64389e5c04c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:22.785515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72878f-aebc-481e-8e1d-6850b8badb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:22.786134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72878f-aebc-481e-8e1d-6850b8badb51 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:22.786403 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72878f-aebc-481e-8e1d-6850b8badb51 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-7f30327a-7625-4b6e-9ef6-64389e5c04c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:22.786501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:22.787340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:22.788180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:22.788180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:22.788180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:22.788180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:22.788180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:22.788180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:22.788180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:22.788180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:22.788180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:22.788180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:22.788180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:22.788180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:22.788811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72878f-aebc-481e-8e1d-6850b8badb51 None admin] PATH_INFO: `/v3/groups` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:22.788811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72878f-aebc-481e-8e1d-6850b8badb51 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:22.788930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 148/305] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:22.788930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30327a-7625-4b6e-9ef6-64389e5c04c2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:22.794521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72878f-aebc-481e-8e1d-6850b8badb51 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:22.824042 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 158/306] 10.222.0.33 () {60 vars in 1134 bytes} [Sun Sep 11 05:16:22 2022] POST /identity/v3/groups => generated 229 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:16:22.824042 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72878f-aebc-481e-8e1d-6850b8badb51 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:22.913383 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30327a-7625-4b6e-9ef6-64389e5c04c2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:22.939738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:22.953379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49804b18-95b5-4ed6-84e4-f58b182b7969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:22.954180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49804b18-95b5-4ed6-84e4-f58b182b7969 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:22.954565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49804b18-95b5-4ed6-84e4-f58b182b7969 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:22.954938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49804b18-95b5-4ed6-84e4-f58b182b7969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:22.955783 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49804b18-95b5-4ed6-84e4-f58b182b7969 None admin] RBAC: Adding query filter params (role.id=8cf91ce8e7dd42aaa2708185cb13a6db, user.id=c8ac14a93b1642429beb53a33ec96ae2, scope.system=all) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:22.956245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49804b18-95b5-4ed6-84e4-f58b182b7969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:22.956366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72878f-aebc-481e-8e1d-6850b8badb51 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:22.958542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49804b18-95b5-4ed6-84e4-f58b182b7969 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:22.972562 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 149/307] 10.222.0.33 () {56 vars in 1308 bytes} [Sun Sep 11 05:16:22 2022] GET /identity/v3/role_assignments?user.id=c8ac14a93b1642429beb53a33ec96ae2&scope.system=all&role.id=8cf91ce8e7dd42aaa2708185cb13a6db => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:22.989259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:22.999342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30147d8-d0d2-4407-b024-942487e4ce72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:22.999981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30147d8-d0d2-4407-b024-942487e4ce72 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:23.000293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30147d8-d0d2-4407-b024-942487e4ce72 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:23.000569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30147d8-d0d2-4407-b024-942487e4ce72 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:23.000761 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7766c-9b16-4e2b-a04f-d3e0acadc36f None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:23.001046 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7766c-9b16-4e2b-a04f-d3e0acadc36f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:23.001251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30147d8-d0d2-4407-b024-942487e4ce72 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:23.001352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7766c-9b16-4e2b-a04f-d3e0acadc36f None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:23.001558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30147d8-d0d2-4407-b024-942487e4ce72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:23.002945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 150/308] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:16:23.003072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30147d8-d0d2-4407-b024-942487e4ce72 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:23.009890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3941e-0d1b-4176-8632-6e0de0364884 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:23.010200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3941e-0d1b-4176-8632-6e0de0364884 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:23.010438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3941e-0d1b-4176-8632-6e0de0364884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:23.015245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 159/309] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:23.020933 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30147d8-d0d2-4407-b024-942487e4ce72 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:23.031902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5cc3941e-0d1b-4176-8632-6e0de0364884 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:23.033194 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5cc3941e-0d1b-4176-8632-6e0de0364884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:23.128600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 151/310] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:23.152018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30147d8-d0d2-4407-b024-942487e4ce72 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:23.188039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089a3e8-08db-4124-a1b1-08e21e310659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:23.189043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089a3e8-08db-4124-a1b1-08e21e310659 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:23.189537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089a3e8-08db-4124-a1b1-08e21e310659 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:23.190070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089a3e8-08db-4124-a1b1-08e21e310659 None admin] PATH_INFO: `/v3/users/a825a3c7a1da417e9db6805e133b8f01` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:23.191669 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089a3e8-08db-4124-a1b1-08e21e310659 None admin] RBAC: Authorizing `identity:get_user(user_id=a825a3c7a1da417e9db6805e133b8f01)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:23.193348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089a3e8-08db-4124-a1b1-08e21e310659 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:23.195519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 160/311] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:23 2022] GET /identity/v3/users/a825a3c7a1da417e9db6805e133b8f01 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:23.201367 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3941e-0d1b-4176-8632-6e0de0364884 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:23.314815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3941e-0d1b-4176-8632-6e0de0364884 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:23.334143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:23.353044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb4adb-9ecd-4b2a-a7f3-b3d3c563a47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:23.353708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4adb-9ecd-4b2a-a7f3-b3d3c563a47f None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:23.353989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4adb-9ecd-4b2a-a7f3-b3d3c563a47f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:23.354265 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4adb-9ecd-4b2a-a7f3-b3d3c563a47f None admin] PATH_INFO: `/v3/system/users/a825a3c7a1da417e9db6805e133b8f01/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:23.355677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb4adb-9ecd-4b2a-a7f3-b3d3c563a47f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a825a3c7a1da417e9db6805e133b8f01, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:23.357095 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb4adb-9ecd-4b2a-a7f3-b3d3c563a47f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:23.370820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 152/312] 10.222.0.33 () {58 vars in 1334 bytes} [Sun Sep 11 05:16:23 2022] PUT /identity/v3/system/users/a825a3c7a1da417e9db6805e133b8f01/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:16:23.420047 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089a3e8-08db-4124-a1b1-08e21e310659 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:23.485511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:23.535786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089a3e8-08db-4124-a1b1-08e21e310659 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:23.575286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74225d59-b420-4ada-b157-d6d3f8e7d58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:23.576129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74225d59-b420-4ada-b157-d6d3f8e7d58a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:23.576129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74225d59-b420-4ada-b157-d6d3f8e7d58a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:23.576380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74225d59-b420-4ada-b157-d6d3f8e7d58a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:23.587190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74225d59-b420-4ada-b157-d6d3f8e7d58a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-74225d59-b420-4ada-b157-d6d3f8e7d58a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:23.592356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:23.593227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:23.594009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:23.594009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:23.594009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:23.594009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:23.594009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:23.594009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:23.594009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:23.594009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:23.594009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:23.594009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:23.594009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:23.594009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:23.594009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 161/313] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:23.601696 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb4adb-9ecd-4b2a-a7f3-b3d3c563a47f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:23.711302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb4adb-9ecd-4b2a-a7f3-b3d3c563a47f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:23.748538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fa3aa-2c25-4f26-8897-506ec28cd970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:23.749157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fa3aa-2c25-4f26-8897-506ec28cd970 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:23.749339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fa3aa-2c25-4f26-8897-506ec28cd970 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:23.749616 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fa3aa-2c25-4f26-8897-506ec28cd970 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:23.750285 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94fa3aa-2c25-4f26-8897-506ec28cd970 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:23.750586 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94fa3aa-2c25-4f26-8897-506ec28cd970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:23.751960 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94fa3aa-2c25-4f26-8897-506ec28cd970 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:23.761746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 153/314] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:23.767380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74225d59-b420-4ada-b157-d6d3f8e7d58a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:23.880663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74225d59-b420-4ada-b157-d6d3f8e7d58a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:23.904027 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:23.923026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830cf17-7e6f-4c61-a258-f88916b1879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:23.923652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830cf17-7e6f-4c61-a258-f88916b1879b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:23.923926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830cf17-7e6f-4c61-a258-f88916b1879b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:23.924220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830cf17-7e6f-4c61-a258-f88916b1879b None admin] PATH_INFO: `/v3/users/f04ba63f7a2d496caa03eacf0ab332b5` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:23.925901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830cf17-7e6f-4c61-a258-f88916b1879b None admin] RBAC: Authorizing `identity:get_user(user_id=f04ba63f7a2d496caa03eacf0ab332b5)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:23.927655 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830cf17-7e6f-4c61-a258-f88916b1879b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:23.929857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 162/315] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:23 2022] GET /identity/v3/users/f04ba63f7a2d496caa03eacf0ab332b5 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:23.935183 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fa3aa-2c25-4f26-8897-506ec28cd970 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:23.983863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:24.045356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fa3aa-2c25-4f26-8897-506ec28cd970 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:24.080786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd1de8-7450-448c-97cd-0ba533b8d7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:24.081936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd1de8-7450-448c-97cd-0ba533b8d7ad None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:24.082326 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd1de8-7450-448c-97cd-0ba533b8d7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:24.082588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd1de8-7450-448c-97cd-0ba533b8d7ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:24.154393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd1de8-7450-448c-97cd-0ba533b8d7ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:24.155935 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd1de8-7450-448c-97cd-0ba533b8d7ad None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:24.203003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:24.261454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 154/316] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:24.266773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830cf17-7e6f-4c61-a258-f88916b1879b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:24.377580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830cf17-7e6f-4c61-a258-f88916b1879b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:24.412936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f4910-4cf2-49ec-9932-124d6e6216b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:24.413575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f4910-4cf2-49ec-9932-124d6e6216b1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:24.413856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f4910-4cf2-49ec-9932-124d6e6216b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:24.414174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f4910-4cf2-49ec-9932-124d6e6216b1 None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:24.415308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f4910-4cf2-49ec-9932-124d6e6216b1 None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:24.416788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f4910-4cf2-49ec-9932-124d6e6216b1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:24.418628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 163/317] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:24 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:24.424528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd1de8-7450-448c-97cd-0ba533b8d7ad None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:24.528285 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd1de8-7450-448c-97cd-0ba533b8d7ad None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:24.563969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2b85e-3036-43cb-8037-d789c1842d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:24.564667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2b85e-3036-43cb-8037-d789c1842d20 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:24.564953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2b85e-3036-43cb-8037-d789c1842d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:24.565225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2b85e-3036-43cb-8037-d789c1842d20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:24.566010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2b85e-3036-43cb-8037-d789c1842d20 None admin] RBAC: Adding query filter params (role.id=8cf91ce8e7dd42aaa2708185cb13a6db, scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17, user.id=f04ba63f7a2d496caa03eacf0ab332b5) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:24.566315 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2b85e-3036-43cb-8037-d789c1842d20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:24.567775 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2b85e-3036-43cb-8037-d789c1842d20 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:24.574123 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 155/318] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:24 2022] GET /identity/v3/role_assignments?user.id=f04ba63f7a2d496caa03eacf0ab332b5&scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17&role.id=8cf91ce8e7dd42aaa2708185cb13a6db => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:25.189128 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79758702-191b-451c-bbba-d547c32be38c None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:25.189716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79758702-191b-451c-bbba-d547c32be38c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:25.189716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79758702-191b-451c-bbba-d547c32be38c None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:25.191631 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 164/319] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:25.197238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96bfbc-c462-410d-ba8e-cb79b254ebd8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:25.197669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96bfbc-c462-410d-ba8e-cb79b254ebd8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:25.197786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96bfbc-c462-410d-ba8e-cb79b254ebd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:25.216405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8d96bfbc-c462-410d-ba8e-cb79b254ebd8 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:25.217607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8d96bfbc-c462-410d-ba8e-cb79b254ebd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:25.219574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:25.273401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7a7ea-efa3-4d01-9f9a-4db893fc71a7 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:25.273772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7a7ea-efa3-4d01-9f9a-4db893fc71a7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:25.274108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7a7ea-efa3-4d01-9f9a-4db893fc71a7 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:25.275635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 165/320] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:16:25.280737 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d3918-96f1-4202-ad51-189f2237fec4 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:25.281024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d3918-96f1-4202-ad51-189f2237fec4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:25.281299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d3918-96f1-4202-ad51-189f2237fec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:25.290522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 156/321] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:25.299681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-766d3918-96f1-4202-ad51-189f2237fec4 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:25.301189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-766d3918-96f1-4202-ad51-189f2237fec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:25.349251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:25.369837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 166/322] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:25.559099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96bfbc-c462-410d-ba8e-cb79b254ebd8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:25.611282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e576d-2c6e-4b9f-98f5-98b251c019cd None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:25.611619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e576d-2c6e-4b9f-98f5-98b251c019cd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:25.611841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e576d-2c6e-4b9f-98f5-98b251c019cd None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:25.613532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 167/323] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:25.618168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6589ba-c7ce-4e2b-9c79-879ec89ce0b2 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:25.618448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6589ba-c7ce-4e2b-9c79-879ec89ce0b2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:25.618717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6589ba-c7ce-4e2b-9c79-879ec89ce0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:25.637531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ba6589ba-c7ce-4e2b-9c79-879ec89ce0b2 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:25.638726 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ba6589ba-c7ce-4e2b-9c79-879ec89ce0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:25.663393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96bfbc-c462-410d-ba8e-cb79b254ebd8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:25.697305 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eae82-3a83-47a2-b135-23e0fa432146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:25.697937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eae82-3a83-47a2-b135-23e0fa432146 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:25.698219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eae82-3a83-47a2-b135-23e0fa432146 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:25.698499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eae82-3a83-47a2-b135-23e0fa432146 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:25.705225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:25.709095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 168/324] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:25.709939 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6589ba-c7ce-4e2b-9c79-879ec89ce0b2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:25.716712 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0eae82-3a83-47a2-b135-23e0fa432146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-0e0eae82-3a83-47a2-b135-23e0fa432146 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:25.721624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:25.722522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:25.723356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:25.723356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:25.723356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:25.723356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:25.723356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:25.723356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:25.723356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:25.723356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:25.723356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:25.723356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:25.723356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:25.723356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:25.723356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 157/325] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:25.728498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eae82-3a83-47a2-b135-23e0fa432146 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:25.818001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6589ba-c7ce-4e2b-9c79-879ec89ce0b2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:25.831149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:25.845662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eae82-3a83-47a2-b135-23e0fa432146 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:25.856294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d8cc8-7f0a-40cc-907b-de407d18c66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:25.856925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d8cc8-7f0a-40cc-907b-de407d18c66b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:25.857198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d8cc8-7f0a-40cc-907b-de407d18c66b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:25.857463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d8cc8-7f0a-40cc-907b-de407d18c66b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:25.858635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d8cc8-7f0a-40cc-907b-de407d18c66b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:25.860140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d8cc8-7f0a-40cc-907b-de407d18c66b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:25.861744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 169/326] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:25.867954 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d8cc8-7f0a-40cc-907b-de407d18c66b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:25.885123 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96112b5b-3f31-4be8-b3db-48200fda9674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:25.885729 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96112b5b-3f31-4be8-b3db-48200fda9674 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:25.886003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96112b5b-3f31-4be8-b3db-48200fda9674 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:25.886274 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96112b5b-3f31-4be8-b3db-48200fda9674 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:25.886937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96112b5b-3f31-4be8-b3db-48200fda9674 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:25.887242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96112b5b-3f31-4be8-b3db-48200fda9674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:25.888654 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96112b5b-3f31-4be8-b3db-48200fda9674 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:25.896932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 158/327] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:25.902326 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96112b5b-3f31-4be8-b3db-48200fda9674 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:25.968421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:25.975628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d8cc8-7f0a-40cc-907b-de407d18c66b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:26.009994 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96112b5b-3f31-4be8-b3db-48200fda9674 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:26.010462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2541d-7b82-4bb9-80f7-94dc2643bf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:26.011066 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2541d-7b82-4bb9-80f7-94dc2643bf20 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:26.011349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2541d-7b82-4bb9-80f7-94dc2643bf20 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:26.011646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2541d-7b82-4bb9-80f7-94dc2643bf20 None admin] PATH_INFO: `/v3/groups` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:26.012304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2541d-7b82-4bb9-80f7-94dc2643bf20 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:26.013729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2541d-7b82-4bb9-80f7-94dc2643bf20 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:26.040032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 170/328] 10.222.0.33 () {60 vars in 1134 bytes} [Sun Sep 11 05:16:25 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:16:26.040032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2541d-7b82-4bb9-80f7-94dc2643bf20 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:26.045160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405823e6-711f-42b4-86aa-e7446cc2ce78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:26.045818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405823e6-711f-42b4-86aa-e7446cc2ce78 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:26.046099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405823e6-711f-42b4-86aa-e7446cc2ce78 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:26.046374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405823e6-711f-42b4-86aa-e7446cc2ce78 None admin] PATH_INFO: `/v3/users/c8ac14a93b1642429beb53a33ec96ae2` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:26.047613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405823e6-711f-42b4-86aa-e7446cc2ce78 None admin] RBAC: Authorizing `identity:get_user(user_id=c8ac14a93b1642429beb53a33ec96ae2)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:26.049173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405823e6-711f-42b4-86aa-e7446cc2ce78 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:26.056495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 159/329] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:25 2022] GET /identity/v3/users/c8ac14a93b1642429beb53a33ec96ae2 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:26.057662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405823e6-711f-42b4-86aa-e7446cc2ce78 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:26.089855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:26.163932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2541d-7b82-4bb9-80f7-94dc2643bf20 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:26.175225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405823e6-711f-42b4-86aa-e7446cc2ce78 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:26.202163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57905f-2641-4214-96cb-db19716edee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:26.203287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57905f-2641-4214-96cb-db19716edee6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:26.203564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57905f-2641-4214-96cb-db19716edee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:26.203834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57905f-2641-4214-96cb-db19716edee6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:26.206121 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:26.215910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57905f-2641-4214-96cb-db19716edee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:26.216591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc473c7f-4ca1-4781-bf49-ecbff058c7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:26.217613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc473c7f-4ca1-4781-bf49-ecbff058c7f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:26.217613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc473c7f-4ca1-4781-bf49-ecbff058c7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:26.217767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc473c7f-4ca1-4781-bf49-ecbff058c7f9 None admin] PATH_INFO: `/v3/system/users/c8ac14a93b1642429beb53a33ec96ae2/roles/8cf91ce8e7dd42aaa2708185cb13a6db` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:26.219180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc473c7f-4ca1-4781-bf49-ecbff058c7f9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c8ac14a93b1642429beb53a33ec96ae2, role_id=8cf91ce8e7dd42aaa2708185cb13a6db)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:26.220628 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc473c7f-4ca1-4781-bf49-ecbff058c7f9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-fd57905f-2641-4214-96cb-db19716edee6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:26.221194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:26.222247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:26.223180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:26.223180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:26.223180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:26.223180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:26.223180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:26.223180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:26.223180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:26.223180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:26.223180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:26.223180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:26.223180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:26.223180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:26.223180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 171/330] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:26.228415 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 160/331] 10.222.0.33 () {58 vars in 1334 bytes} [Sun Sep 11 05:16:26 2022] PUT /identity/v3/system/users/c8ac14a93b1642429beb53a33ec96ae2/roles/8cf91ce8e7dd42aaa2708185cb13a6db => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:16:26.228678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57905f-2641-4214-96cb-db19716edee6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:26.346739 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:26.352164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57905f-2641-4214-96cb-db19716edee6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:26.390311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aaea0-73fb-4b3a-a16f-3cee3b2f9e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:26.390940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aaea0-73fb-4b3a-a16f-3cee3b2f9e1c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:26.391265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aaea0-73fb-4b3a-a16f-3cee3b2f9e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:26.391539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aaea0-73fb-4b3a-a16f-3cee3b2f9e1c None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:26.392259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aaea0-73fb-4b3a-a16f-3cee3b2f9e1c None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:26.392560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aaea0-73fb-4b3a-a16f-3cee3b2f9e1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:26.394702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aaea0-73fb-4b3a-a16f-3cee3b2f9e1c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:26.405199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 172/332] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:26.411006 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc473c7f-4ca1-4781-bf49-ecbff058c7f9 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:26.521845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc473c7f-4ca1-4781-bf49-ecbff058c7f9 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:26.558939 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69481ed8-12fd-420f-b05c-99ffb16a5903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:26.559645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69481ed8-12fd-420f-b05c-99ffb16a5903 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:26.559926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69481ed8-12fd-420f-b05c-99ffb16a5903 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:26.560246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69481ed8-12fd-420f-b05c-99ffb16a5903 None admin] PATH_INFO: `/v3/users/a825a3c7a1da417e9db6805e133b8f01` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:26.561441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69481ed8-12fd-420f-b05c-99ffb16a5903 None admin] RBAC: Authorizing `identity:get_user(user_id=a825a3c7a1da417e9db6805e133b8f01)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:26.562894 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69481ed8-12fd-420f-b05c-99ffb16a5903 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:26.564837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 161/333] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:26 2022] GET /identity/v3/users/a825a3c7a1da417e9db6805e133b8f01 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:26.571361 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aaea0-73fb-4b3a-a16f-3cee3b2f9e1c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:26.684591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aaea0-73fb-4b3a-a16f-3cee3b2f9e1c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:26.726972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a8736-a34b-4d52-8750-e1a255455b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:26.727599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a8736-a34b-4d52-8750-e1a255455b9d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:26.727876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a8736-a34b-4d52-8750-e1a255455b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:26.728177 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a8736-a34b-4d52-8750-e1a255455b9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:26.728915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a8736-a34b-4d52-8750-e1a255455b9d None admin] RBAC: Adding query filter params (role.id=d9f6babaca2d494c9a04f0aca1271af8, user.id=a825a3c7a1da417e9db6805e133b8f01, scope.system=all) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:26.729218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a8736-a34b-4d52-8750-e1a255455b9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:26.731039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a8736-a34b-4d52-8750-e1a255455b9d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:26.750267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 173/334] 10.222.0.33 () {56 vars in 1308 bytes} [Sun Sep 11 05:16:26 2022] GET /identity/v3/role_assignments?user.id=a825a3c7a1da417e9db6805e133b8f01&scope.system=all&role.id=d9f6babaca2d494c9a04f0aca1271af8 => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:28.598069 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7b770-d0dc-46f9-bc3d-c6e3f0d6998a None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:28.598624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7b770-d0dc-46f9-bc3d-c6e3f0d6998a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:28.598624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7b770-d0dc-46f9-bc3d-c6e3f0d6998a None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:28.600281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 162/335] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:28.606189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213810b8-64a0-4434-8abe-06164d3f132f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:28.607126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213810b8-64a0-4434-8abe-06164d3f132f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:28.607621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213810b8-64a0-4434-8abe-06164d3f132f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:28.637034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-213810b8-64a0-4434-8abe-06164d3f132f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:28.638608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-213810b8-64a0-4434-8abe-06164d3f132f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:28.655784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:28.718267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 174/336] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:28.751218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1b6d2-4885-4231-a109-046da4d5a4ad None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:28.751501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1b6d2-4885-4231-a109-046da4d5a4ad None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:28.751946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1b6d2-4885-4231-a109-046da4d5a4ad None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:28.754056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 163/337] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:28.767815 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f1565-735a-439f-85b7-b4661c7ff107 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:28.768124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f1565-735a-439f-85b7-b4661c7ff107 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:28.768389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f1565-735a-439f-85b7-b4661c7ff107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:28.791893 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f21f1565-735a-439f-85b7-b4661c7ff107 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:28.793119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f21f1565-735a-439f-85b7-b4661c7ff107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:28.878701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 175/338] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:29.155709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6e4dc-bd8a-4cdb-a729-21fd9e12973b None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:29.155709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6e4dc-bd8a-4cdb-a729-21fd9e12973b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:29.155709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6e4dc-bd8a-4cdb-a729-21fd9e12973b None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:29.156415 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 164/339] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:29.162331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9c66f-ece6-418e-8fb2-98edfc998556 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:29.162876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9c66f-ece6-418e-8fb2-98edfc998556 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:29.163410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9c66f-ece6-418e-8fb2-98edfc998556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:29.187790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d8b9c66f-ece6-418e-8fb2-98edfc998556 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:29.189392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d8b9c66f-ece6-418e-8fb2-98edfc998556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:29.281273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 176/340] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:29.318395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6e4dc-bd8a-4cdb-a729-21fd9e12973b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:29.400446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:29.449079 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6e4dc-bd8a-4cdb-a729-21fd9e12973b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:29.487824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5c066-e0d4-45a1-84d1-269293642364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:29.488563 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5c066-e0d4-45a1-84d1-269293642364 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:29.488842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5c066-e0d4-45a1-84d1-269293642364 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:29.489164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5c066-e0d4-45a1-84d1-269293642364 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:29.502862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5c066-e0d4-45a1-84d1-269293642364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-4df5c066-e0d4-45a1-84d1-269293642364 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:29.510065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:29.510974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:29.511779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:29.511779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:29.511779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:29.511779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:29.511779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:29.511779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:29.511779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:29.511779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:29.511779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:29.511779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:29.511779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:29.511779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:29.511779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 165/341] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:29.520238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9c66f-ece6-418e-8fb2-98edfc998556 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:29.537820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:29.587818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5c066-e0d4-45a1-84d1-269293642364 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:29.675170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9c66f-ece6-418e-8fb2-98edfc998556 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:29.736552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69bd8cb-2301-487f-9025-52c024f3044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:29.737622 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bd8cb-2301-487f-9025-52c024f3044b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:29.738354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bd8cb-2301-487f-9025-52c024f3044b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:29.738847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bd8cb-2301-487f-9025-52c024f3044b None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:29.739773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69bd8cb-2301-487f-9025-52c024f3044b None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:29.740282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69bd8cb-2301-487f-9025-52c024f3044b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:29.741994 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69bd8cb-2301-487f-9025-52c024f3044b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:29.758169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 177/342] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:29.759329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69bd8cb-2301-487f-9025-52c024f3044b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:29.798471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5c066-e0d4-45a1-84d1-269293642364 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:29.836717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:29.856480 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efb561-0a4d-4ec7-8f98-5a0b2c414048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:29.857518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efb561-0a4d-4ec7-8f98-5a0b2c414048 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:29.858018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efb561-0a4d-4ec7-8f98-5a0b2c414048 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:29.858486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efb561-0a4d-4ec7-8f98-5a0b2c414048 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:29.869851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efb561-0a4d-4ec7-8f98-5a0b2c414048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:29.874384 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-40efb561-0a4d-4ec7-8f98-5a0b2c414048 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:29.875883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:29.876883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:29.877829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:29.877829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:29.877829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:29.877829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:29.877829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:29.877829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:29.877829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:29.877829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:29.877829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:29.877829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:29.877829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 05:16:29.877829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:29.882236 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 166/343] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:29.883308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efb561-0a4d-4ec7-8f98-5a0b2c414048 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:29.913723 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69bd8cb-2301-487f-9025-52c024f3044b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:29.953959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c2a86-7d26-4d34-ab43-717d8151ef49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:29.954597 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c2a86-7d26-4d34-ab43-717d8151ef49 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:29.954873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c2a86-7d26-4d34-ab43-717d8151ef49 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:29.955143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c2a86-7d26-4d34-ab43-717d8151ef49 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:29.967409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c2a86-7d26-4d34-ab43-717d8151ef49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-fe4c2a86-7d26-4d34-ab43-717d8151ef49 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:29.973336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:29.974294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:29.975148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:29.975148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:29.975148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:29.975148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:29.975148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:29.975148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:29.975148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:29.975148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:29.975148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:29.975148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:29.975148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:29.975148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:29.975148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 178/344] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:16:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:29.975853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4c4d8-5617-4ddd-92f6-6a1063eb0538 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:29.975853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4c4d8-5617-4ddd-92f6-6a1063eb0538 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:29.980623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4c4d8-5617-4ddd-92f6-6a1063eb0538 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:29.982127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 179/345] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:29.983005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4c4d8-5617-4ddd-92f6-6a1063eb0538 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:30.003108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efb561-0a4d-4ec7-8f98-5a0b2c414048 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:30.040565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509820e8-71fe-4651-a4d7-bca1b3b14f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:30.041540 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509820e8-71fe-4651-a4d7-bca1b3b14f9a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:30.042032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509820e8-71fe-4651-a4d7-bca1b3b14f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:30.042512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509820e8-71fe-4651-a4d7-bca1b3b14f9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:30.093063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:30.105200 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4c4d8-5617-4ddd-92f6-6a1063eb0538 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:30.115821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509820e8-71fe-4651-a4d7-bca1b3b14f9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:30.117638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509820e8-71fe-4651-a4d7-bca1b3b14f9a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:30.139089 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:30.146703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096ac65-c754-4faf-9ddc-dab85795c4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:30.147598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096ac65-c754-4faf-9ddc-dab85795c4f1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:30.148186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096ac65-c754-4faf-9ddc-dab85795c4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:30.148643 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096ac65-c754-4faf-9ddc-dab85795c4f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:30.149569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096ac65-c754-4faf-9ddc-dab85795c4f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:30.150050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096ac65-c754-4faf-9ddc-dab85795c4f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:30.151786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096ac65-c754-4faf-9ddc-dab85795c4f1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:30.162430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 180/346] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:30.163666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096ac65-c754-4faf-9ddc-dab85795c4f1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:30.230364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 167/347] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:30.231776 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649274-e98c-4372-a0d1-bd0f5e772f40 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:30.232325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649274-e98c-4372-a0d1-bd0f5e772f40 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:30.232814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649274-e98c-4372-a0d1-bd0f5e772f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:30.251418 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e0649274-e98c-4372-a0d1-bd0f5e772f40 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:30.252997 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e0649274-e98c-4372-a0d1-bd0f5e772f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:30.281521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096ac65-c754-4faf-9ddc-dab85795c4f1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:30.326475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e5dd1-b64d-4345-9474-8d8e32c9016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:30.327134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e5dd1-b64d-4345-9474-8d8e32c9016d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:30.327410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e5dd1-b64d-4345-9474-8d8e32c9016d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:30.327674 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e5dd1-b64d-4345-9474-8d8e32c9016d None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:30.328372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e5dd1-b64d-4345-9474-8d8e32c9016d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:30.328672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e5dd1-b64d-4345-9474-8d8e32c9016d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:30.330398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e5dd1-b64d-4345-9474-8d8e32c9016d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:30.332105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 168/348] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:30.332747 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649274-e98c-4372-a0d1-bd0f5e772f40 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:30.342147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 181/349] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:30.342996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e5dd1-b64d-4345-9474-8d8e32c9016d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:30.348908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:30.410539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:30.465917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e5dd1-b64d-4345-9474-8d8e32c9016d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:30.469968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649274-e98c-4372-a0d1-bd0f5e772f40 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:30.507513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9605b-f328-46c9-8a61-81ca2c00e5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:30.508997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9605b-f328-46c9-8a61-81ca2c00e5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:30.508997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9605b-f328-46c9-8a61-81ca2c00e5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:30.508997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9605b-f328-46c9-8a61-81ca2c00e5a6 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:30.510523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9605b-f328-46c9-8a61-81ca2c00e5a6 None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:30.512246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9605b-f328-46c9-8a61-81ca2c00e5a6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:30.514279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 182/350] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:30 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:30.515373 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495153e7-0825-45cd-9f8b-bb1c7fdfebc8 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:30.515783 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495153e7-0825-45cd-9f8b-bb1c7fdfebc8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:30.516111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495153e7-0825-45cd-9f8b-bb1c7fdfebc8 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:30.517309 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 183/351] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:30.518077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495153e7-0825-45cd-9f8b-bb1c7fdfebc8 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:30.518350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4eb3cc-c69f-4fe4-bf69-41b3344228c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:30.519473 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eb3cc-c69f-4fe4-bf69-41b3344228c6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:30.520055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eb3cc-c69f-4fe4-bf69-41b3344228c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:30.521033 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eb3cc-c69f-4fe4-bf69-41b3344228c6 None admin] PATH_INFO: `/v3/users/c8ac14a93b1642429beb53a33ec96ae2` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:30.523436 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4eb3cc-c69f-4fe4-bf69-41b3344228c6 None admin] RBAC: Authorizing `identity:get_user(user_id=c8ac14a93b1642429beb53a33ec96ae2)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:30.525161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4eb3cc-c69f-4fe4-bf69-41b3344228c6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:30.527270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 169/352] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:16:30 2022] GET /identity/v3/users/c8ac14a93b1642429beb53a33ec96ae2 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:30.529231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb1f8c-67db-49f8-95a0-561071f03e59 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:30.529740 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb1f8c-67db-49f8-95a0-561071f03e59 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:30.530224 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb1f8c-67db-49f8-95a0-561071f03e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:30.556696 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-fcfb1f8c-67db-49f8-95a0-561071f03e59 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:30.558931 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fcfb1f8c-67db-49f8-95a0-561071f03e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:30.652235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495153e7-0825-45cd-9f8b-bb1c7fdfebc8 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:30.656472 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 170/353] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:30.657693 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb1f8c-67db-49f8-95a0-561071f03e59 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:30.678565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:30.684368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:30.697104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54204b-b3a6-403b-a081-85524fcb5e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:30.697717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54204b-b3a6-403b-a081-85524fcb5e94 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:30.697995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54204b-b3a6-403b-a081-85524fcb5e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:30.698262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54204b-b3a6-403b-a081-85524fcb5e94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:30.791556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54204b-b3a6-403b-a081-85524fcb5e94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:30.793408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54204b-b3a6-403b-a081-85524fcb5e94 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:30.819567 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb1f8c-67db-49f8-95a0-561071f03e59 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:30.860335 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3e5be-bac9-4d3a-a802-26bee310f9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:30.861020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3e5be-bac9-4d3a-a802-26bee310f9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:30.861307 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3e5be-bac9-4d3a-a802-26bee310f9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:30.861576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3e5be-bac9-4d3a-a802-26bee310f9e1 None admin] PATH_INFO: `/v3/groups/9d67e7db62f34c61a75e00abbc599884` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:30.868624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3e5be-bac9-4d3a-a802-26bee310f9e1 None admin] RBAC: Authorizing `identity:get_group(group_id=9d67e7db62f34c61a75e00abbc599884)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:30.870073 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3e5be-bac9-4d3a-a802-26bee310f9e1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:30.871858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 171/354] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:30 2022] GET /identity/v3/groups/9d67e7db62f34c61a75e00abbc599884 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:30.872568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3e5be-bac9-4d3a-a802-26bee310f9e1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:30.936564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 184/355] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:30.938156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54204b-b3a6-403b-a081-85524fcb5e94 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:30.961587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:30.969856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:30.996228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3e5be-bac9-4d3a-a802-26bee310f9e1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:31.037622 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939bca9-3057-4238-9c98-053a8807f6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:31.038231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939bca9-3057-4238-9c98-053a8807f6bf None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:31.038510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939bca9-3057-4238-9c98-053a8807f6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:31.038778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939bca9-3057-4238-9c98-053a8807f6bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:31.039627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939bca9-3057-4238-9c98-053a8807f6bf None admin] RBAC: Adding query filter params (role.id=8cf91ce8e7dd42aaa2708185cb13a6db, user.id=c8ac14a93b1642429beb53a33ec96ae2, scope.system=all) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:31.039934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939bca9-3057-4238-9c98-053a8807f6bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:31.041409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939bca9-3057-4238-9c98-053a8807f6bf None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:31.054997 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 172/356] 10.222.0.33 () {56 vars in 1308 bytes} [Sun Sep 11 05:16:30 2022] GET /identity/v3/role_assignments?user.id=c8ac14a93b1642429beb53a33ec96ae2&scope.system=all&role.id=8cf91ce8e7dd42aaa2708185cb13a6db => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:31.056181 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939bca9-3057-4238-9c98-053a8807f6bf None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:31.068709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54204b-b3a6-403b-a081-85524fcb5e94 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:31.127337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf782e4-d485-45ee-9cf5-7fbba22a51d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:31.127906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf782e4-d485-45ee-9cf5-7fbba22a51d5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:31.129131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf782e4-d485-45ee-9cf5-7fbba22a51d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:31.129567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf782e4-d485-45ee-9cf5-7fbba22a51d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:31.144944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf782e4-d485-45ee-9cf5-7fbba22a51d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5bf782e4-d485-45ee-9cf5-7fbba22a51d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:31.149953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:31.150907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 185/357] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:31.152264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf782e4-d485-45ee-9cf5-7fbba22a51d5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:31.221521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939bca9-3057-4238-9c98-053a8807f6bf None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:31.257152 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:31.265136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53da24-a0cd-4102-9c87-4fe4fadc7c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:31.266399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53da24-a0cd-4102-9c87-4fe4fadc7c9c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:31.266938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53da24-a0cd-4102-9c87-4fe4fadc7c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:31.267395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:31.267531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53da24-a0cd-4102-9c87-4fe4fadc7c9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:31.268371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53da24-a0cd-4102-9c87-4fe4fadc7c9c None admin] RBAC: Adding query filter params (group.id=9d67e7db62f34c61a75e00abbc599884, role.id=d9f6babaca2d494c9a04f0aca1271af8, scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:31.269110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53da24-a0cd-4102-9c87-4fe4fadc7c9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:31.270693 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53da24-a0cd-4102-9c87-4fe4fadc7c9c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:31.278902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 173/358] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 05:16:31 2022] GET /identity/v3/role_assignments?group.id=9d67e7db62f34c61a75e00abbc599884&scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3&role.id=d9f6babaca2d494c9a04f0aca1271af8 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:31.279440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53da24-a0cd-4102-9c87-4fe4fadc7c9c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:31.298506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf782e4-d485-45ee-9cf5-7fbba22a51d5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:31.358992 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b5804-bba5-445f-90a3-504e2b97c0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:31.359767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b5804-bba5-445f-90a3-504e2b97c0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:31.360070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b5804-bba5-445f-90a3-504e2b97c0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:31.360716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b5804-bba5-445f-90a3-504e2b97c0c3 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:31.362408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b5804-bba5-445f-90a3-504e2b97c0c3 None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:31.364919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b5804-bba5-445f-90a3-504e2b97c0c3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:31.367581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 186/359] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:31 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:31.368745 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b5804-bba5-445f-90a3-504e2b97c0c3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:31.437322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53da24-a0cd-4102-9c87-4fe4fadc7c9c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:31.495752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df7d28-09f9-49cc-870f-a64dc6fdca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:31.496698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df7d28-09f9-49cc-870f-a64dc6fdca9a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:31.496985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df7d28-09f9-49cc-870f-a64dc6fdca9a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:31.497256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df7d28-09f9-49cc-870f-a64dc6fdca9a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:31.510614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df7d28-09f9-49cc-870f-a64dc6fdca9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:31.515369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b5804-bba5-445f-90a3-504e2b97c0c3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-62df7d28-09f9-49cc-870f-a64dc6fdca9a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:31.516338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:31.517244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 174/360] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:16:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:31.518391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df7d28-09f9-49cc-870f-a64dc6fdca9a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:31.568170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2c8e8-0c8b-4507-a3f8-96f5bb0ab7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:31.569281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2c8e8-0c8b-4507-a3f8-96f5bb0ab7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:31.569771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2c8e8-0c8b-4507-a3f8-96f5bb0ab7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:31.570347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2c8e8-0c8b-4507-a3f8-96f5bb0ab7b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:31.571226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2c8e8-0c8b-4507-a3f8-96f5bb0ab7b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:31.571741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2c8e8-0c8b-4507-a3f8-96f5bb0ab7b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:31.573434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2c8e8-0c8b-4507-a3f8-96f5bb0ab7b5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:31.584363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:31.585826 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 187/361] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:31.587417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac1137-7ee3-40a2-b0bb-c71c95fcbffc None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:31.587951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac1137-7ee3-40a2-b0bb-c71c95fcbffc None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:31.588458 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac1137-7ee3-40a2-b0bb-c71c95fcbffc None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:31.590661 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 188/362] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:31.591645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac1137-7ee3-40a2-b0bb-c71c95fcbffc None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:31.609703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:31.650403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df7d28-09f9-49cc-870f-a64dc6fdca9a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:31.693755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837546ca-ecdb-42a0-ae7c-423423621462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:31.694968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837546ca-ecdb-42a0-ae7c-423423621462 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:31.695700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837546ca-ecdb-42a0-ae7c-423423621462 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:31.696184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837546ca-ecdb-42a0-ae7c-423423621462 None admin] PATH_INFO: `/v3/groups/9d67e7db62f34c61a75e00abbc599884` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:31.697567 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837546ca-ecdb-42a0-ae7c-423423621462 None admin] RBAC: Authorizing `identity:get_group(group_id=9d67e7db62f34c61a75e00abbc599884)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:31.699177 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837546ca-ecdb-42a0-ae7c-423423621462 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:31.700937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 175/363] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:31 2022] GET /identity/v3/groups/9d67e7db62f34c61a75e00abbc599884 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:31.707757 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837546ca-ecdb-42a0-ae7c-423423621462 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:31.726392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac1137-7ee3-40a2-b0bb-c71c95fcbffc None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:31.774775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505bde5-06bd-48e0-80ec-4450ac723ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:31.775394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505bde5-06bd-48e0-80ec-4450ac723ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:31.775672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505bde5-06bd-48e0-80ec-4450ac723ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:31.775939 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505bde5-06bd-48e0-80ec-4450ac723ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:31.776660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505bde5-06bd-48e0-80ec-4450ac723ba1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:31.776988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505bde5-06bd-48e0-80ec-4450ac723ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:31.778364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505bde5-06bd-48e0-80ec-4450ac723ba1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:31.787617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 189/364] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:31.789543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2950f0-3828-4b72-95fc-576e03d7a32f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:31.789828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2950f0-3828-4b72-95fc-576e03d7a32f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:31.790100 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2950f0-3828-4b72-95fc-576e03d7a32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:31.814009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6b2950f0-3828-4b72-95fc-576e03d7a32f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:31.815451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6b2950f0-3828-4b72-95fc-576e03d7a32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:31.841408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837546ca-ecdb-42a0-ae7c-423423621462 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:31.872988 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:31.891004 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e671da0-6141-446a-9c8e-d99398a747f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:31.891941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e671da0-6141-446a-9c8e-d99398a747f3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:31.892981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e671da0-6141-446a-9c8e-d99398a747f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:31.892981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e671da0-6141-446a-9c8e-d99398a747f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:31.906406 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 190/365] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:31.907239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2950f0-3828-4b72-95fc-576e03d7a32f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:31.922644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:31.980977 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e671da0-6141-446a-9c8e-d99398a747f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:31.982353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e671da0-6141-446a-9c8e-d99398a747f3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:32.035883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2950f0-3828-4b72-95fc-576e03d7a32f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:32.086032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600ab94-e02a-4584-a8f3-9e02ab8b6219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:32.086674 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600ab94-e02a-4584-a8f3-9e02ab8b6219 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:32.086948 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600ab94-e02a-4584-a8f3-9e02ab8b6219 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:32.087222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600ab94-e02a-4584-a8f3-9e02ab8b6219 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:32.088019 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600ab94-e02a-4584-a8f3-9e02ab8b6219 None admin] RBAC: Adding query filter params (group.id=9d67e7db62f34c61a75e00abbc599884, role.id=6c6c7fa19ee342da915c90895e707620, scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:32.088347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600ab94-e02a-4584-a8f3-9e02ab8b6219 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:32.089977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600ab94-e02a-4584-a8f3-9e02ab8b6219 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:32.096988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 191/366] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 05:16:31 2022] GET /identity/v3/role_assignments?group.id=9d67e7db62f34c61a75e00abbc599884&scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3&role.id=6c6c7fa19ee342da915c90895e707620 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:32.097834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600ab94-e02a-4584-a8f3-9e02ab8b6219 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:32.127475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 176/367] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:32.137221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e671da0-6141-446a-9c8e-d99398a747f3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:32.180147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:32.206965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:32.252671 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600ab94-e02a-4584-a8f3-9e02ab8b6219 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:32.292351 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e671da0-6141-446a-9c8e-d99398a747f3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:32.301011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05e999-0e00-4296-8f51-ee75220e6a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:32.301646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05e999-0e00-4296-8f51-ee75220e6a5e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:32.301930 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05e999-0e00-4296-8f51-ee75220e6a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:32.302203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05e999-0e00-4296-8f51-ee75220e6a5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:32.340322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ce246-76de-41d4-b88a-6e7d74fcca80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:32.341038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ce246-76de-41d4-b88a-6e7d74fcca80 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:32.341361 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ce246-76de-41d4-b88a-6e7d74fcca80 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:32.341586 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ce246-76de-41d4-b88a-6e7d74fcca80 None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:32.343125 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ce246-76de-41d4-b88a-6e7d74fcca80 None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:32.344663 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ce246-76de-41d4-b88a-6e7d74fcca80 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:32.346535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 177/368] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:32 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:32.347400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ce246-76de-41d4-b88a-6e7d74fcca80 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:32.420359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05e999-0e00-4296-8f51-ee75220e6a5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:32.423885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05e999-0e00-4296-8f51-ee75220e6a5e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:32.481237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:32.498495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ce246-76de-41d4-b88a-6e7d74fcca80 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:32.527257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:32.551197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003c067-dd3c-46c3-ada2-1a432d614126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:32.551846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003c067-dd3c-46c3-ada2-1a432d614126 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:32.552157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003c067-dd3c-46c3-ada2-1a432d614126 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:32.552432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003c067-dd3c-46c3-ada2-1a432d614126 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:32.567611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003c067-dd3c-46c3-ada2-1a432d614126 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:32.571921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 192/369] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:32.573033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05e999-0e00-4296-8f51-ee75220e6a5e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-0003c067-dd3c-46c3-ada2-1a432d614126 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:32.573897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:32.574785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:32.575594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:32.575594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:32.575594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:32.575594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:32.575594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:32.575594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:32.575594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:32.575594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:32.575594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:32.575594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:32.575594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:32.575594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:32.575594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 178/370] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:16:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:32.578070 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003c067-dd3c-46c3-ada2-1a432d614126 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:32.703719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05e999-0e00-4296-8f51-ee75220e6a5e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:32.724596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003c067-dd3c-46c3-ada2-1a432d614126 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:32.756492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7220928-cb81-4025-8f33-6bf38928a9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:32.757031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7220928-cb81-4025-8f33-6bf38928a9df None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:32.757247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7220928-cb81-4025-8f33-6bf38928a9df None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:32.757447 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7220928-cb81-4025-8f33-6bf38928a9df None admin] PATH_INFO: `/v3/groups/9d67e7db62f34c61a75e00abbc599884` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:32.758742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7220928-cb81-4025-8f33-6bf38928a9df None admin] RBAC: Authorizing `identity:get_group(group_id=9d67e7db62f34c61a75e00abbc599884)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:32.760220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7220928-cb81-4025-8f33-6bf38928a9df None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:32.762866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 193/371] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:32 2022] GET /identity/v3/groups/9d67e7db62f34c61a75e00abbc599884 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:32.763702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7220928-cb81-4025-8f33-6bf38928a9df None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:32.772957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e29cc9-ca5d-4b1b-94fb-fce06351269f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:32.773926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e29cc9-ca5d-4b1b-94fb-fce06351269f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:32.774427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e29cc9-ca5d-4b1b-94fb-fce06351269f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:32.774899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e29cc9-ca5d-4b1b-94fb-fce06351269f None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:32.776338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e29cc9-ca5d-4b1b-94fb-fce06351269f None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:32.778072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e29cc9-ca5d-4b1b-94fb-fce06351269f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:32.780286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 179/372] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:32 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:32.781259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e29cc9-ca5d-4b1b-94fb-fce06351269f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:32.811610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:32.822516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:32.908607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7220928-cb81-4025-8f33-6bf38928a9df None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:32.948439 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e29cc9-ca5d-4b1b-94fb-fce06351269f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:32.961213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f0dfd-c7f8-4284-acd6-105dcc695073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:32.961213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f0dfd-c7f8-4284-acd6-105dcc695073 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:32.961213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f0dfd-c7f8-4284-acd6-105dcc695073 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:32.961715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f0dfd-c7f8-4284-acd6-105dcc695073 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:32.962497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f0dfd-c7f8-4284-acd6-105dcc695073 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:32.962921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f0dfd-c7f8-4284-acd6-105dcc695073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:32.965070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f0dfd-c7f8-4284-acd6-105dcc695073 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:32.976622 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 194/373] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:32.977600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f0dfd-c7f8-4284-acd6-105dcc695073 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:32.995581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b2ae3-4198-46bc-a945-2118e896d091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:32.997043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b2ae3-4198-46bc-a945-2118e896d091 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:32.997572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b2ae3-4198-46bc-a945-2118e896d091 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:32.998133 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b2ae3-4198-46bc-a945-2118e896d091 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:32.999112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b2ae3-4198-46bc-a945-2118e896d091 None admin] RBAC: Adding query filter params (group.id=9d67e7db62f34c61a75e00abbc599884, role.id=d9f6babaca2d494c9a04f0aca1271af8, scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:32.999639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b2ae3-4198-46bc-a945-2118e896d091 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:33.001354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b2ae3-4198-46bc-a945-2118e896d091 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:33.008635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 180/374] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 05:16:32 2022] GET /identity/v3/role_assignments?group.id=9d67e7db62f34c61a75e00abbc599884&scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17&role.id=d9f6babaca2d494c9a04f0aca1271af8 => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:33.009820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b2ae3-4198-46bc-a945-2118e896d091 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:33.112029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:33.116136 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f0dfd-c7f8-4284-acd6-105dcc695073 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:33.129719 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:33.157104 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b2ae3-4198-46bc-a945-2118e896d091 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:33.170396 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce5de4-ac49-42b7-8b30-908409dc6123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:33.171059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce5de4-ac49-42b7-8b30-908409dc6123 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:33.171338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce5de4-ac49-42b7-8b30-908409dc6123 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:33.171609 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce5de4-ac49-42b7-8b30-908409dc6123 None admin] PATH_INFO: `/v3/groups/9d67e7db62f34c61a75e00abbc599884` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:33.173087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce5de4-ac49-42b7-8b30-908409dc6123 None admin] RBAC: Authorizing `identity:get_group(group_id=9d67e7db62f34c61a75e00abbc599884)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:33.174551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce5de4-ac49-42b7-8b30-908409dc6123 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:33.179448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 195/375] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:32 2022] GET /identity/v3/groups/9d67e7db62f34c61a75e00abbc599884 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:33.187359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce5de4-ac49-42b7-8b30-908409dc6123 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:33.237877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e275d3a-ad34-4bba-89da-313a10c93ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:33.238570 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e275d3a-ad34-4bba-89da-313a10c93ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:33.238851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e275d3a-ad34-4bba-89da-313a10c93ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:33.239125 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e275d3a-ad34-4bba-89da-313a10c93ac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:33.353669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e275d3a-ad34-4bba-89da-313a10c93ac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:33.355330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e275d3a-ad34-4bba-89da-313a10c93ac4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:33.374476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce5de4-ac49-42b7-8b30-908409dc6123 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:33.434565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2330090-d59b-4723-9420-98179c241ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:33.435554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2330090-d59b-4723-9420-98179c241ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:33.436084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2330090-d59b-4723-9420-98179c241ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:33.436582 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2330090-d59b-4723-9420-98179c241ad0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:33.438083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2330090-d59b-4723-9420-98179c241ad0 None admin] RBAC: Adding query filter params (group.id=9d67e7db62f34c61a75e00abbc599884, role.id=6c6c7fa19ee342da915c90895e707620, scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:33.439791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2330090-d59b-4723-9420-98179c241ad0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:33.442269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2330090-d59b-4723-9420-98179c241ad0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:33.454748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 196/376] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 05:16:33 2022] GET /identity/v3/role_assignments?group.id=9d67e7db62f34c61a75e00abbc599884&scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17&role.id=6c6c7fa19ee342da915c90895e707620 => generated 260 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:33.497030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:33.527555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 181/377] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:33.532969 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2330090-d59b-4723-9420-98179c241ad0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:33.601666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:33.724804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2330090-d59b-4723-9420-98179c241ad0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:33.760926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abdf3f2-712f-4339-baec-b12717a80d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:33.762330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdf3f2-712f-4339-baec-b12717a80d6b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:33.762818 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdf3f2-712f-4339-baec-b12717a80d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:33.763290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdf3f2-712f-4339-baec-b12717a80d6b None admin] PATH_INFO: `/v3/projects/4c05020216314cd9bc11af667152fcc9` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:33.764797 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abdf3f2-712f-4339-baec-b12717a80d6b None admin] RBAC: Authorizing `identity:get_project(project_id=4c05020216314cd9bc11af667152fcc9)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:33.766504 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abdf3f2-712f-4339-baec-b12717a80d6b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:33.769383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 197/378] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:33 2022] GET /identity/v3/projects/4c05020216314cd9bc11af667152fcc9 => generated 344 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:33.774882 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e275d3a-ad34-4bba-89da-313a10c93ac4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:33.935873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e275d3a-ad34-4bba-89da-313a10c93ac4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:33.979120 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09373bbc-0714-483b-896b-ce4b8a573d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:33.979725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09373bbc-0714-483b-896b-ce4b8a573d3a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:33.979947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09373bbc-0714-483b-896b-ce4b8a573d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:33.980230 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09373bbc-0714-483b-896b-ce4b8a573d3a None admin] PATH_INFO: `/v3/groups/568077d393d240c5b673198e30a3bd45` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:33.991786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09373bbc-0714-483b-896b-ce4b8a573d3a None admin] RBAC: Authorizing `identity:get_group(group_id=568077d393d240c5b673198e30a3bd45)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:33.993283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09373bbc-0714-483b-896b-ce4b8a573d3a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:33.995281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 182/379] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:33 2022] GET /identity/v3/groups/568077d393d240c5b673198e30a3bd45 => generated 229 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:34.000876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abdf3f2-712f-4339-baec-b12717a80d6b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:34.032044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564a004-ea62-48d6-a412-6c7df061fce5 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:34.032044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564a004-ea62-48d6-a412-6c7df061fce5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:34.032044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564a004-ea62-48d6-a412-6c7df061fce5 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:34.032044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 183/380] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:34.038080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8efba-691f-42fd-9737-39830b5d22a1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:34.038353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8efba-691f-42fd-9737-39830b5d22a1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:34.038624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8efba-691f-42fd-9737-39830b5d22a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:34.067108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-bfe8efba-691f-42fd-9737-39830b5d22a1 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:34.068379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bfe8efba-691f-42fd-9737-39830b5d22a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:34.119691 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:34.156652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 184/381] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:34.179485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:34.183008 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abdf3f2-712f-4339-baec-b12717a80d6b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:34.238269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59174bd5-97e4-4861-a848-a78fdc07e18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:34.239020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59174bd5-97e4-4861-a848-a78fdc07e18e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:34.239356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59174bd5-97e4-4861-a848-a78fdc07e18e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:34.239633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59174bd5-97e4-4861-a848-a78fdc07e18e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:34.240392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59174bd5-97e4-4861-a848-a78fdc07e18e None admin] RBAC: Adding query filter params (group.id=568077d393d240c5b673198e30a3bd45, role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=4c05020216314cd9bc11af667152fcc9) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:34.240702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59174bd5-97e4-4861-a848-a78fdc07e18e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:34.242141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59174bd5-97e4-4861-a848-a78fdc07e18e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:34.249884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 198/382] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 05:16:33 2022] GET /identity/v3/role_assignments?group.id=568077d393d240c5b673198e30a3bd45&scope.project.id=4c05020216314cd9bc11af667152fcc9&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:34.578855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8efba-691f-42fd-9737-39830b5d22a1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:34.703011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8efba-691f-42fd-9737-39830b5d22a1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:34.748965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a8861-31de-4a10-8e01-851e324c0743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:34.749594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a8861-31de-4a10-8e01-851e324c0743 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:34.749874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a8861-31de-4a10-8e01-851e324c0743 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:34.750141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a8861-31de-4a10-8e01-851e324c0743 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:34.761785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a8861-31de-4a10-8e01-851e324c0743 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-3c4a8861-31de-4a10-8e01-851e324c0743 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:34.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:34.767489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:34.768341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:34.768341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:34.768341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:34.768341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:34.768341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:34.768341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:34.768341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:34.768341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:34.768341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:34.768341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:34.768341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:34.768341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:34.769762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 185/383] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:34.774764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59174bd5-97e4-4861-a848-a78fdc07e18e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:34.887946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59174bd5-97e4-4861-a848-a78fdc07e18e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:34.923077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447458b-8952-4262-8f7a-c40a3897ec63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:34.923687 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447458b-8952-4262-8f7a-c40a3897ec63 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:34.923964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447458b-8952-4262-8f7a-c40a3897ec63 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:34.924258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447458b-8952-4262-8f7a-c40a3897ec63 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:34.924969 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447458b-8952-4262-8f7a-c40a3897ec63 None admin] RBAC: Adding query filter params (name=member) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:34.925431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447458b-8952-4262-8f7a-c40a3897ec63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:34.927078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447458b-8952-4262-8f7a-c40a3897ec63 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:34.940117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 199/384] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:34.942421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a8861-31de-4a10-8e01-851e324c0743 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:34.985619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:35.067388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a8861-31de-4a10-8e01-851e324c0743 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:35.110247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1c175-4d66-42e4-93d5-1579f632b8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:35.110877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1c175-4d66-42e4-93d5-1579f632b8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:35.111150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1c175-4d66-42e4-93d5-1579f632b8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:35.111418 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1c175-4d66-42e4-93d5-1579f632b8d3 None admin] PATH_INFO: `/v3/groups/9d67e7db62f34c61a75e00abbc599884` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:35.113282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1c175-4d66-42e4-93d5-1579f632b8d3 None admin] RBAC: Authorizing `identity:get_group(group_id=9d67e7db62f34c61a75e00abbc599884)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:35.115059 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1c175-4d66-42e4-93d5-1579f632b8d3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:35.120102 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 186/385] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:34 2022] GET /identity/v3/groups/9d67e7db62f34c61a75e00abbc599884 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:35.122723 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447458b-8952-4262-8f7a-c40a3897ec63 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:35.156451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:35.196561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdb47a-84b2-4249-bd00-80db666a86b0 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:35.196868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdb47a-84b2-4249-bd00-80db666a86b0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:35.197106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdb47a-84b2-4249-bd00-80db666a86b0 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:35.198742 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 187/386] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:16:35.209707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7167d-e384-4756-a02c-b04cc63c33b0 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:35.209992 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7167d-e384-4756-a02c-b04cc63c33b0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:35.210417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7167d-e384-4756-a02c-b04cc63c33b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:35.233286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7ce7167d-e384-4756-a02c-b04cc63c33b0 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:35.235281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7ce7167d-e384-4756-a02c-b04cc63c33b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:35.248659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447458b-8952-4262-8f7a-c40a3897ec63 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:35.319433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00a01b-4c0f-472b-871f-56e1cfe2c0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:35.320484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00a01b-4c0f-472b-871f-56e1cfe2c0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:35.320984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00a01b-4c0f-472b-871f-56e1cfe2c0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:35.321493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00a01b-4c0f-472b-871f-56e1cfe2c0f9 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:35.322892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00a01b-4c0f-472b-871f-56e1cfe2c0f9 None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:35.324587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00a01b-4c0f-472b-871f-56e1cfe2c0f9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:35.328975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 200/387] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:35 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:35.335624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00a01b-4c0f-472b-871f-56e1cfe2c0f9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:35.371095 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 188/388] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:35.447424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:35.455216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00a01b-4c0f-472b-871f-56e1cfe2c0f9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:35.491375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4f5f7-c4e7-4105-906c-bd33589fcc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:35.492474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4f5f7-c4e7-4105-906c-bd33589fcc27 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:35.492970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4f5f7-c4e7-4105-906c-bd33589fcc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:35.493618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4f5f7-c4e7-4105-906c-bd33589fcc27 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3/groups/9d67e7db62f34c61a75e00abbc599884/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:35.495761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4f5f7-c4e7-4105-906c-bd33589fcc27 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3, group_id=9d67e7db62f34c61a75e00abbc599884, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:35.497417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4f5f7-c4e7-4105-906c-bd33589fcc27 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:35.523648 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 201/389] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 11 05:16:35 2022] PUT /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3/groups/9d67e7db62f34c61a75e00abbc599884/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:16:35.685135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d3a93-9459-41d8-9009-dfeff5bbc355 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:35.685847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d3a93-9459-41d8-9009-dfeff5bbc355 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:35.686338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d3a93-9459-41d8-9009-dfeff5bbc355 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:35.688256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 189/390] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:35.693927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa486b39-6b7b-47ea-a4d3-fcf37cf31077 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:35.694217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa486b39-6b7b-47ea-a4d3-fcf37cf31077 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:35.694494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa486b39-6b7b-47ea-a4d3-fcf37cf31077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:35.713726 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fa486b39-6b7b-47ea-a4d3-fcf37cf31077 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:35.714919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fa486b39-6b7b-47ea-a4d3-fcf37cf31077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:35.806607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 202/391] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:35.877014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d3a93-9459-41d8-9009-dfeff5bbc355 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:35.919868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:36.001661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d3a93-9459-41d8-9009-dfeff5bbc355 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:36.041481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ddd08-c178-4d0b-8729-70bc00578db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:36.042112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ddd08-c178-4d0b-8729-70bc00578db5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:36.042450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ddd08-c178-4d0b-8729-70bc00578db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:36.042717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ddd08-c178-4d0b-8729-70bc00578db5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:36.054261 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ddd08-c178-4d0b-8729-70bc00578db5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-1f6ddd08-c178-4d0b-8729-70bc00578db5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:36.059255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:36.060202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:36.061018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:36.061018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:36.061018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:36.061018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:36.061018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:36.061018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:36.061018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:36.061018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:36.061018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:36.061018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:36.061018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:36.061018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:36.062530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 190/392] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:16:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:36.067016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa486b39-6b7b-47ea-a4d3-fcf37cf31077 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:36.149137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ddd08-c178-4d0b-8729-70bc00578db5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:36.164194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:36.180696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa486b39-6b7b-47ea-a4d3-fcf37cf31077 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:36.227072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7617765-079f-44df-8034-18b7785394a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:36.228185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7617765-079f-44df-8034-18b7785394a7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:36.228710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7617765-079f-44df-8034-18b7785394a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:36.229183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7617765-079f-44df-8034-18b7785394a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:36.230743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7617765-079f-44df-8034-18b7785394a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:36.231543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7617765-079f-44df-8034-18b7785394a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:36.233362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7617765-079f-44df-8034-18b7785394a7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:36.246352 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 203/393] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:36.253898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7617765-079f-44df-8034-18b7785394a7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:36.292066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ddd08-c178-4d0b-8729-70bc00578db5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:36.294155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:36.333324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbdf07-ef08-4490-9d54-0b926e98249e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:36.333970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbdf07-ef08-4490-9d54-0b926e98249e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:36.334247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbdf07-ef08-4490-9d54-0b926e98249e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:36.334520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbdf07-ef08-4490-9d54-0b926e98249e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:36.345228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbdf07-ef08-4490-9d54-0b926e98249e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-cedbdf07-ef08-4490-9d54-0b926e98249e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:36.350690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:36.351600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:36.352421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:36.352421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:36.352421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:36.352421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:36.352421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:36.352421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:36.352421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:36.352421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:36.352421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:36.352421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:36.352421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:36.352421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:36.352421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 191/394] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:36.358820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbdf07-ef08-4490-9d54-0b926e98249e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:36.400642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7617765-079f-44df-8034-18b7785394a7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:36.477516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a88e67-50f2-4d0a-856f-4da393a01d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:36.478473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a88e67-50f2-4d0a-856f-4da393a01d4a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:36.478931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a88e67-50f2-4d0a-856f-4da393a01d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:36.479345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a88e67-50f2-4d0a-856f-4da393a01d4a None admin] PATH_INFO: `/v3/groups/9d67e7db62f34c61a75e00abbc599884` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:36.483848 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbdf07-ef08-4490-9d54-0b926e98249e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:36.484674 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a88e67-50f2-4d0a-856f-4da393a01d4a None admin] RBAC: Authorizing `identity:get_group(group_id=9d67e7db62f34c61a75e00abbc599884)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:36.486513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a88e67-50f2-4d0a-856f-4da393a01d4a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:36.488574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 204/395] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:36 2022] GET /identity/v3/groups/9d67e7db62f34c61a75e00abbc599884 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:36.494489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a88e67-50f2-4d0a-856f-4da393a01d4a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:36.534223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2beaf8-4bd0-42f5-a3ef-bae82c1c5f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:36.535253 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2beaf8-4bd0-42f5-a3ef-bae82c1c5f62 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:36.535735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2beaf8-4bd0-42f5-a3ef-bae82c1c5f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:36.536277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2beaf8-4bd0-42f5-a3ef-bae82c1c5f62 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:36.537483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:36.538422 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2beaf8-4bd0-42f5-a3ef-bae82c1c5f62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:36.538998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2beaf8-4bd0-42f5-a3ef-bae82c1c5f62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:36.540752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2beaf8-4bd0-42f5-a3ef-bae82c1c5f62 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:36.552416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 192/396] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:36.557506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2beaf8-4bd0-42f5-a3ef-bae82c1c5f62 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:36.580400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:36.651704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a88e67-50f2-4d0a-856f-4da393a01d4a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:36.665049 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2beaf8-4bd0-42f5-a3ef-bae82c1c5f62 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:36.692088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f536e-19c9-4d47-bbd6-3877eb5bcc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:36.693113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f536e-19c9-4d47-bbd6-3877eb5bcc89 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:36.693675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f536e-19c9-4d47-bbd6-3877eb5bcc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:36.694150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f536e-19c9-4d47-bbd6-3877eb5bcc89 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:36.695683 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f536e-19c9-4d47-bbd6-3877eb5bcc89 None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:36.698008 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f536e-19c9-4d47-bbd6-3877eb5bcc89 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:36.700457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 205/397] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:36 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:36.701880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2d613-b920-4651-b1cc-7f855ffe8df3 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:36.702365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2d613-b920-4651-b1cc-7f855ffe8df3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:36.702855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2d613-b920-4651-b1cc-7f855ffe8df3 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:36.704583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 206/398] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:36.705238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33ea63-a64b-4a27-91fd-459d566fbed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:36.705882 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33ea63-a64b-4a27-91fd-459d566fbed2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:36.706899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33ea63-a64b-4a27-91fd-459d566fbed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:36.707450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33ea63-a64b-4a27-91fd-459d566fbed2 None admin] PATH_INFO: `/v3/groups/9d67e7db62f34c61a75e00abbc599884` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:36.712247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33ea63-a64b-4a27-91fd-459d566fbed2 None admin] RBAC: Authorizing `identity:get_group(group_id=9d67e7db62f34c61a75e00abbc599884)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:36.712247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33ea63-a64b-4a27-91fd-459d566fbed2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:36.712866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 193/399] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:36 2022] GET /identity/v3/groups/9d67e7db62f34c61a75e00abbc599884 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:36.713806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2d613-b920-4651-b1cc-7f855ffe8df3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:36.714056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92915386-27ca-4a63-811d-44f82b4e6fb7 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:36.714340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92915386-27ca-4a63-811d-44f82b4e6fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:36.714612 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92915386-27ca-4a63-811d-44f82b4e6fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:36.734856 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-92915386-27ca-4a63-811d-44f82b4e6fb7 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:36.736081 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-92915386-27ca-4a63-811d-44f82b4e6fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:36.826974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:36.839065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 194/400] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:36.839065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92915386-27ca-4a63-811d-44f82b4e6fb7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:36.862018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2d613-b920-4651-b1cc-7f855ffe8df3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:36.869174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:36.949971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b9522-9d97-40ea-95ae-ec7c298f61d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:36.950648 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b9522-9d97-40ea-95ae-ec7c298f61d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:36.950929 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b9522-9d97-40ea-95ae-ec7c298f61d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:36.951202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b9522-9d97-40ea-95ae-ec7c298f61d9 None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3/groups/9d67e7db62f34c61a75e00abbc599884/roles/6c6c7fa19ee342da915c90895e707620` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:36.958461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b9522-9d97-40ea-95ae-ec7c298f61d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3, group_id=9d67e7db62f34c61a75e00abbc599884, role_id=6c6c7fa19ee342da915c90895e707620)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:36.960251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b9522-9d97-40ea-95ae-ec7c298f61d9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:36.986872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 207/401] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 11 05:16:36 2022] PUT /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3/groups/9d67e7db62f34c61a75e00abbc599884/roles/6c6c7fa19ee342da915c90895e707620 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:16:36.995946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92915386-27ca-4a63-811d-44f82b4e6fb7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:37.046061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec7400-94eb-4743-b99a-08abd3f1eefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:37.047516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec7400-94eb-4743-b99a-08abd3f1eefb None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:37.048044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec7400-94eb-4743-b99a-08abd3f1eefb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:37.048566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec7400-94eb-4743-b99a-08abd3f1eefb None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:37.050492 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec7400-94eb-4743-b99a-08abd3f1eefb None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:37.052247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec7400-94eb-4743-b99a-08abd3f1eefb None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:37.054171 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 195/402] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:36 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:37.060288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b9522-9d97-40ea-95ae-ec7c298f61d9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:37.197279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec7400-94eb-4743-b99a-08abd3f1eefb None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:37.206166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b9522-9d97-40ea-95ae-ec7c298f61d9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:37.272902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dfbf0-e512-452d-93eb-02e0e782ea5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:37.272902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dfbf0-e512-452d-93eb-02e0e782ea5c None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:37.272902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dfbf0-e512-452d-93eb-02e0e782ea5c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:37.273280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dfbf0-e512-452d-93eb-02e0e782ea5c None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17/groups/9d67e7db62f34c61a75e00abbc599884/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:37.275056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13dfbf0-e512-452d-93eb-02e0e782ea5c None admin] RBAC: Authorizing `identity:create_grant(project_id=7c6f2a4f66134019b9d71c524ab5cb17, group_id=9d67e7db62f34c61a75e00abbc599884, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:37.276527 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13dfbf0-e512-452d-93eb-02e0e782ea5c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:37.291976 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 208/403] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 11 05:16:37 2022] PUT /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17/groups/9d67e7db62f34c61a75e00abbc599884/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:16:37.293378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832cdad-e8fc-4a6b-a3a7-7723ffa53c0a None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:37.293737 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832cdad-e8fc-4a6b-a3a7-7723ffa53c0a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:37.294016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832cdad-e8fc-4a6b-a3a7-7723ffa53c0a None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:37.295247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 209/404] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:37.307608 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:37.310275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d68965-4d7e-469e-8986-334de300ac9c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:37.310880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d68965-4d7e-469e-8986-334de300ac9c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:37.311103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d68965-4d7e-469e-8986-334de300ac9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:37.324027 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:37.333566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec7400-94eb-4743-b99a-08abd3f1eefb None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:37.339104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-39d68965-4d7e-469e-8986-334de300ac9c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:37.340362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-39d68965-4d7e-469e-8986-334de300ac9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:37.391895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de56a35-ef85-4d83-b762-b16162910c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:37.392602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de56a35-ef85-4d83-b762-b16162910c44 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:37.392882 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de56a35-ef85-4d83-b762-b16162910c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:37.393155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de56a35-ef85-4d83-b762-b16162910c44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:37.406751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de56a35-ef85-4d83-b762-b16162910c44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-6de56a35-ef85-4d83-b762-b16162910c44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:37.413380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:37.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:37.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:37.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:37.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:37.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:37.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:37.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:37.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:37.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:37.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:37.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:37.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:37.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:37.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 196/405] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:16:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:37.420685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de56a35-ef85-4d83-b762-b16162910c44 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:37.449224 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 210/406] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:37.562657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de56a35-ef85-4d83-b762-b16162910c44 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:37.600840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639da479-3f23-4001-901d-8cf0ebb71797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:37.601454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639da479-3f23-4001-901d-8cf0ebb71797 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:37.601732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639da479-3f23-4001-901d-8cf0ebb71797 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:37.602004 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639da479-3f23-4001-901d-8cf0ebb71797 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:37.602672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639da479-3f23-4001-901d-8cf0ebb71797 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:37.602973 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639da479-3f23-4001-901d-8cf0ebb71797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:37.604370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639da479-3f23-4001-901d-8cf0ebb71797 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:37.623976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 197/407] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:37.629844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d68965-4d7e-469e-8986-334de300ac9c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:37.792196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d68965-4d7e-469e-8986-334de300ac9c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:37.812043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639da479-3f23-4001-901d-8cf0ebb71797 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:37.812605 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:37.861348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:37.872232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb0787-cb8b-4732-9363-16aacbbaa886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:37.872889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb0787-cb8b-4732-9363-16aacbbaa886 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:37.873172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb0787-cb8b-4732-9363-16aacbbaa886 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:37.873447 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb0787-cb8b-4732-9363-16aacbbaa886 None admin] PATH_INFO: `/v3/groups/9d67e7db62f34c61a75e00abbc599884` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:37.874657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb0787-cb8b-4732-9363-16aacbbaa886 None admin] RBAC: Authorizing `identity:get_group(group_id=9d67e7db62f34c61a75e00abbc599884)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:37.883028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb0787-cb8b-4732-9363-16aacbbaa886 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:37.886535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 211/408] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:37 2022] GET /identity/v3/groups/9d67e7db62f34c61a75e00abbc599884 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:37.890982 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb0787-cb8b-4732-9363-16aacbbaa886 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:38.013406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639da479-3f23-4001-901d-8cf0ebb71797 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:38.049590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb0787-cb8b-4732-9363-16aacbbaa886 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:38.091707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e96325-c2fe-4270-b11d-f2138fda733f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:38.092387 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e96325-c2fe-4270-b11d-f2138fda733f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:38.092900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e96325-c2fe-4270-b11d-f2138fda733f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:38.093185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e96325-c2fe-4270-b11d-f2138fda733f None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:38.094682 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e96325-c2fe-4270-b11d-f2138fda733f None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:38.096169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e96325-c2fe-4270-b11d-f2138fda733f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:38.100263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 212/409] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:37 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:38.108325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74631e7-250e-48ef-bbde-fa4994f98e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:38.109134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74631e7-250e-48ef-bbde-fa4994f98e29 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:38.109319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74631e7-250e-48ef-bbde-fa4994f98e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:38.109591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74631e7-250e-48ef-bbde-fa4994f98e29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:38.112212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e96325-c2fe-4270-b11d-f2138fda733f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:38.132331 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74631e7-250e-48ef-bbde-fa4994f98e29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-d74631e7-250e-48ef-bbde-fa4994f98e29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:38.145499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:38.146629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:38.148142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:38.148142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:38.148142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:38.148142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:38.148142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:38.148142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:38.148142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:38.148142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:38.148142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:38.148142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:38.148142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:38.148142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:38.148142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 198/410] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:16:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:38.157030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74631e7-250e-48ef-bbde-fa4994f98e29 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:38.236606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:38.278747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e96325-c2fe-4270-b11d-f2138fda733f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:38.295070 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:38.365174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f72f3-d4c9-4dc1-82cc-036aab07a658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:38.365782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f72f3-d4c9-4dc1-82cc-036aab07a658 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:38.366064 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f72f3-d4c9-4dc1-82cc-036aab07a658 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:38.366339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f72f3-d4c9-4dc1-82cc-036aab07a658 None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17/groups/9d67e7db62f34c61a75e00abbc599884/roles/6c6c7fa19ee342da915c90895e707620` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:38.370457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f72f3-d4c9-4dc1-82cc-036aab07a658 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c6f2a4f66134019b9d71c524ab5cb17, group_id=9d67e7db62f34c61a75e00abbc599884, role_id=6c6c7fa19ee342da915c90895e707620)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:38.373249 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f72f3-d4c9-4dc1-82cc-036aab07a658 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:38.382184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74631e7-250e-48ef-bbde-fa4994f98e29 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:38.395062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 213/411] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 11 05:16:38 2022] PUT /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17/groups/9d67e7db62f34c61a75e00abbc599884/roles/6c6c7fa19ee342da915c90895e707620 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:16:38.396932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca432bd-7c7f-4d5e-8e07-3222bcd48708 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:38.397269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca432bd-7c7f-4d5e-8e07-3222bcd48708 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:38.397860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca432bd-7c7f-4d5e-8e07-3222bcd48708 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:38.404278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 214/412] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:38.405736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4aa70-4058-492f-b9cb-efe2cab5f0fd None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:38.405736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4aa70-4058-492f-b9cb-efe2cab5f0fd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:38.405736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4aa70-4058-492f-b9cb-efe2cab5f0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:38.446784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-79d4aa70-4058-492f-b9cb-efe2cab5f0fd None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:38.448128 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-79d4aa70-4058-492f-b9cb-efe2cab5f0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:38.458825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b2115-a9f1-439e-a619-9a9447694f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:38.459447 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b2115-a9f1-439e-a619-9a9447694f8d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:38.459732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b2115-a9f1-439e-a619-9a9447694f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:38.460020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b2115-a9f1-439e-a619-9a9447694f8d None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:38.460714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b2115-a9f1-439e-a619-9a9447694f8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:38.461075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b2115-a9f1-439e-a619-9a9447694f8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:38.462794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b2115-a9f1-439e-a619-9a9447694f8d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:38.484781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 199/413] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:38.491502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b2115-a9f1-439e-a619-9a9447694f8d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:38.634962 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 215/414] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:38.733894 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b2115-a9f1-439e-a619-9a9447694f8d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:38.781820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:38.792261 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933eff99-d03d-4a6c-ac6f-ee13c59e8223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:38.792998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933eff99-d03d-4a6c-ac6f-ee13c59e8223 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:38.793282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933eff99-d03d-4a6c-ac6f-ee13c59e8223 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:38.793628 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933eff99-d03d-4a6c-ac6f-ee13c59e8223 None admin] PATH_INFO: `/v3/groups/568077d393d240c5b673198e30a3bd45` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:38.794750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933eff99-d03d-4a6c-ac6f-ee13c59e8223 None admin] RBAC: Authorizing `identity:get_group(group_id=568077d393d240c5b673198e30a3bd45)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:38.796270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933eff99-d03d-4a6c-ac6f-ee13c59e8223 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:38.797844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 200/415] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:38 2022] GET /identity/v3/groups/568077d393d240c5b673198e30a3bd45 => generated 229 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:38.804725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4aa70-4058-492f-b9cb-efe2cab5f0fd None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:38.919911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:38.930640 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933eff99-d03d-4a6c-ac6f-ee13c59e8223 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:38.984303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4aa70-4058-492f-b9cb-efe2cab5f0fd None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:39.062874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39902e4-0a29-4079-964d-ee294c968503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:39.065621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39902e4-0a29-4079-964d-ee294c968503 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:39.065621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39902e4-0a29-4079-964d-ee294c968503 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:39.065621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39902e4-0a29-4079-964d-ee294c968503 None admin] PATH_INFO: `/v3/projects/4c05020216314cd9bc11af667152fcc9` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:39.065621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39902e4-0a29-4079-964d-ee294c968503 None admin] RBAC: Authorizing `identity:get_project(project_id=4c05020216314cd9bc11af667152fcc9)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:39.066886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39902e4-0a29-4079-964d-ee294c968503 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:39.069017 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 216/416] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:38 2022] GET /identity/v3/projects/4c05020216314cd9bc11af667152fcc9 => generated 344 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:39.081761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39902e4-0a29-4079-964d-ee294c968503 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:39.132036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933eff99-d03d-4a6c-ac6f-ee13c59e8223 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:39.207206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2b28b-98e0-4c22-9a14-094a0af9eb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:39.208156 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2b28b-98e0-4c22-9a14-094a0af9eb61 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:39.208577 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2b28b-98e0-4c22-9a14-094a0af9eb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:39.208988 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2b28b-98e0-4c22-9a14-094a0af9eb61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:39.211374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39902e4-0a29-4079-964d-ee294c968503 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:39.236979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2b28b-98e0-4c22-9a14-094a0af9eb61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:39.237644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ddc2b28b-98e0-4c22-9a14-094a0af9eb61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:39.252021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:39.254135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:39.254961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:39.254961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:39.254961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:39.254961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:39.254961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:39.254961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:39.254961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:39.254961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:39.254961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:39.254961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:39.254961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:39.254961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:39.255581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3b23e-4dee-41c6-9343-c0e1474d016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:39.255581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3b23e-4dee-41c6-9343-c0e1474d016d None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:39.255581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3b23e-4dee-41c6-9343-c0e1474d016d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:39.255581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3b23e-4dee-41c6-9343-c0e1474d016d None admin] PATH_INFO: `/v3/projects/4c05020216314cd9bc11af667152fcc9/groups/568077d393d240c5b673198e30a3bd45/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:39.255830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3b23e-4dee-41c6-9343-c0e1474d016d None admin] RBAC: Authorizing `identity:create_grant(project_id=4c05020216314cd9bc11af667152fcc9, group_id=568077d393d240c5b673198e30a3bd45, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:39.255830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3b23e-4dee-41c6-9343-c0e1474d016d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:39.257341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 201/417] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:16:39.263525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2b28b-98e0-4c22-9a14-094a0af9eb61 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:39.280668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 217/418] 10.222.0.33 () {58 vars in 1442 bytes} [Sun Sep 11 05:16:39 2022] PUT /identity/v3/projects/4c05020216314cd9bc11af667152fcc9/groups/568077d393d240c5b673198e30a3bd45/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:16:39.344457 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:39.395038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2b28b-98e0-4c22-9a14-094a0af9eb61 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:39.433770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05acebc-3bdb-4790-a2ec-7b51e5d25a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:39.434399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05acebc-3bdb-4790-a2ec-7b51e5d25a60 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:39.434677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05acebc-3bdb-4790-a2ec-7b51e5d25a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:39.434951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05acebc-3bdb-4790-a2ec-7b51e5d25a60 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:39.435702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05acebc-3bdb-4790-a2ec-7b51e5d25a60 None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:39.436020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05acebc-3bdb-4790-a2ec-7b51e5d25a60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:39.437482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05acebc-3bdb-4790-a2ec-7b51e5d25a60 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:39.455958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 202/419] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:39.462480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3b23e-4dee-41c6-9343-c0e1474d016d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:39.572416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3b23e-4dee-41c6-9343-c0e1474d016d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:39.608580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96737d4-ba00-4dc8-a72a-89487265575b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:39.609197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96737d4-ba00-4dc8-a72a-89487265575b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:39.609479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96737d4-ba00-4dc8-a72a-89487265575b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:39.609752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96737d4-ba00-4dc8-a72a-89487265575b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:39.669203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:39.683928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96737d4-ba00-4dc8-a72a-89487265575b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:39.685409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96737d4-ba00-4dc8-a72a-89487265575b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:39.824543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 218/420] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:39.829915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05acebc-3bdb-4790-a2ec-7b51e5d25a60 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:39.954949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05acebc-3bdb-4790-a2ec-7b51e5d25a60 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:39.972433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:39.991586 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cdc3f-ce84-4e31-bfb8-f1bf43e7be2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:39.992251 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cdc3f-ce84-4e31-bfb8-f1bf43e7be2b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:39.992531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cdc3f-ce84-4e31-bfb8-f1bf43e7be2b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:39.992804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cdc3f-ce84-4e31-bfb8-f1bf43e7be2b None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:39.993925 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cdc3f-ce84-4e31-bfb8-f1bf43e7be2b None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:39.995381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cdc3f-ce84-4e31-bfb8-f1bf43e7be2b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:39.997339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 203/421] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:39 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:39.999928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed062f35-3ecc-4e6f-a0e8-5eb5ad55b975 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:40.000410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed062f35-3ecc-4e6f-a0e8-5eb5ad55b975 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:40.000611 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed062f35-3ecc-4e6f-a0e8-5eb5ad55b975 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:40.002063 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 219/422] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:40.002974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cdc3f-ce84-4e31-bfb8-f1bf43e7be2b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:40.007907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98f218-d98c-4f48-873c-87ab9c264bad None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:40.008212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98f218-d98c-4f48-873c-87ab9c264bad None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:40.008478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98f218-d98c-4f48-873c-87ab9c264bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:40.057055 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9a98f218-d98c-4f48-873c-87ab9c264bad None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:40.060199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9a98f218-d98c-4f48-873c-87ab9c264bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:40.130877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cdc3f-ce84-4e31-bfb8-f1bf43e7be2b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:40.181686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0356c8-d2a2-4f52-b0da-2548bb0051d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:40.182333 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0356c8-d2a2-4f52-b0da-2548bb0051d6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:40.182588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0356c8-d2a2-4f52-b0da-2548bb0051d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:40.182854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0356c8-d2a2-4f52-b0da-2548bb0051d6 None admin] PATH_INFO: `/v3/groups/9d67e7db62f34c61a75e00abbc599884` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:40.184742 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0356c8-d2a2-4f52-b0da-2548bb0051d6 None admin] RBAC: Authorizing `identity:get_group(group_id=9d67e7db62f34c61a75e00abbc599884)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:40.186187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0356c8-d2a2-4f52-b0da-2548bb0051d6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:40.194541 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 204/423] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:40 2022] GET /identity/v3/groups/9d67e7db62f34c61a75e00abbc599884 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:40.197724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0356c8-d2a2-4f52-b0da-2548bb0051d6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:40.236130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 220/424] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:40.263278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876252df-77d0-4376-b05e-e4740111a1d6 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:40.263557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876252df-77d0-4376-b05e-e4740111a1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:40.263829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876252df-77d0-4376-b05e-e4740111a1d6 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:40.265191 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 221/425] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:16:40.270882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc85ce7-504f-40a1-a7fb-fb2ed7f60caa None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:40.271155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc85ce7-504f-40a1-a7fb-fb2ed7f60caa None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:40.271425 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc85ce7-504f-40a1-a7fb-fb2ed7f60caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:40.295553 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ccc85ce7-504f-40a1-a7fb-fb2ed7f60caa None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:40.296816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ccc85ce7-504f-40a1-a7fb-fb2ed7f60caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:40.310021 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:40.312459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:40.391976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0356c8-d2a2-4f52-b0da-2548bb0051d6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:40.415401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 222/426] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:40.442185 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7292fd-9fef-40fd-bd9a-d20ca4c916ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:40.442792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7292fd-9fef-40fd-bd9a-d20ca4c916ff None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:40.443070 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7292fd-9fef-40fd-bd9a-d20ca4c916ff None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:40.443339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7292fd-9fef-40fd-bd9a-d20ca4c916ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:40.444138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7292fd-9fef-40fd-bd9a-d20ca4c916ff None admin] RBAC: Adding query filter params (group.id=9d67e7db62f34c61a75e00abbc599884, role.id=d9f6babaca2d494c9a04f0aca1271af8, scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:40.444445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7292fd-9fef-40fd-bd9a-d20ca4c916ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:40.445948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7292fd-9fef-40fd-bd9a-d20ca4c916ff None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:40.457555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 205/427] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 05:16:40 2022] GET /identity/v3/role_assignments?group.id=9d67e7db62f34c61a75e00abbc599884&scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3&role.id=d9f6babaca2d494c9a04f0aca1271af8 => generated 611 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:40.635277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc85ce7-504f-40a1-a7fb-fb2ed7f60caa None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:40.754984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc85ce7-504f-40a1-a7fb-fb2ed7f60caa None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:40.769805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7292fd-9fef-40fd-bd9a-d20ca4c916ff None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:40.808651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2613baa-5da1-4169-b8ac-d527447e5f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:40.809091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2613baa-5da1-4169-b8ac-d527447e5f4a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:40.809363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2613baa-5da1-4169-b8ac-d527447e5f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:40.809627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2613baa-5da1-4169-b8ac-d527447e5f4a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:40.821415 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2613baa-5da1-4169-b8ac-d527447e5f4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-c2613baa-5da1-4169-b8ac-d527447e5f4a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:40.826705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:40.827604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:40.828773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:40.828773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:40.828773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:40.828773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:40.828773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:40.828773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:40.828773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:40.828773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:40.828773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:40.828773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:40.828773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:40.828773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:40.832532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 223/428] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:16:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:40.839014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2613baa-5da1-4169-b8ac-d527447e5f4a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:40.851901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:40.932709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7292fd-9fef-40fd-bd9a-d20ca4c916ff None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:40.952306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:40.964891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2613baa-5da1-4169-b8ac-d527447e5f4a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:40.975053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36530f-27de-4322-8d7e-057879d5fac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:40.976080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36530f-27de-4322-8d7e-057879d5fac1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:40.976507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36530f-27de-4322-8d7e-057879d5fac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:40.976918 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36530f-27de-4322-8d7e-057879d5fac1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:40.988587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36530f-27de-4322-8d7e-057879d5fac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-5c36530f-27de-4322-8d7e-057879d5fac1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:40.994471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:40.996269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:40.996269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:40.996269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:40.996269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:40.996269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:40.996269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:40.996269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:40.996269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:40.996269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:40.996269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:40.996269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 05:16:40.996269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:40.998943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 206/429] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:41.005828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36530f-27de-4322-8d7e-057879d5fac1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:41.014004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc5d79-e491-4b1d-bbc7-6e065e24ba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:41.014656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc5d79-e491-4b1d-bbc7-6e065e24ba5e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:41.015204 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc5d79-e491-4b1d-bbc7-6e065e24ba5e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:41.015481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc5d79-e491-4b1d-bbc7-6e065e24ba5e None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:41.016727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc5d79-e491-4b1d-bbc7-6e065e24ba5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:41.016727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc5d79-e491-4b1d-bbc7-6e065e24ba5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:41.017900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc5d79-e491-4b1d-bbc7-6e065e24ba5e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:41.032135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 224/430] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:41.035299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc5d79-e491-4b1d-bbc7-6e065e24ba5e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:41.124353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:41.138196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36530f-27de-4322-8d7e-057879d5fac1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:41.164923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc5d79-e491-4b1d-bbc7-6e065e24ba5e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:41.179179 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bfe7c-0e86-431f-b121-273cbded0a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:41.180825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bfe7c-0e86-431f-b121-273cbded0a8f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:41.181310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bfe7c-0e86-431f-b121-273cbded0a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:41.181734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bfe7c-0e86-431f-b121-273cbded0a8f None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:41.183088 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0bfe7c-0e86-431f-b121-273cbded0a8f None admin] RBAC: Adding query filter params (name=member) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:41.183605 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0bfe7c-0e86-431f-b121-273cbded0a8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:41.185428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0bfe7c-0e86-431f-b121-273cbded0a8f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:41.197789 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 207/431] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:16:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:41.203736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bfe7c-0e86-431f-b121-273cbded0a8f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:41.205195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97f32f-c1ae-486b-b08f-1014a415eeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:41.205957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97f32f-c1ae-486b-b08f-1014a415eeae None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:41.206309 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97f32f-c1ae-486b-b08f-1014a415eeae None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:41.206654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97f32f-c1ae-486b-b08f-1014a415eeae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:41.297559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97f32f-c1ae-486b-b08f-1014a415eeae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:41.300265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97f32f-c1ae-486b-b08f-1014a415eeae None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:41.320394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:41.329342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bfe7c-0e86-431f-b121-273cbded0a8f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:41.375131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92352c50-0745-4faf-b4ef-025b8cec3bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:41.375740 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92352c50-0745-4faf-b4ef-025b8cec3bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:41.376056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92352c50-0745-4faf-b4ef-025b8cec3bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:41.376326 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92352c50-0745-4faf-b4ef-025b8cec3bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:41.387228 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:41.426677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 225/432] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 05:16:41.428373 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f469f32-1335-4e9e-b51d-6bc852e96944 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:41.428864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f469f32-1335-4e9e-b51d-6bc852e96944 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:41.429358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f469f32-1335-4e9e-b51d-6bc852e96944 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:41.431229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 226/433] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:41.432321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f469f32-1335-4e9e-b51d-6bc852e96944 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:41.460366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92352c50-0745-4faf-b4ef-025b8cec3bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:41.461867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92352c50-0745-4faf-b4ef-025b8cec3bd2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:41.506466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:41.539951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f469f32-1335-4e9e-b51d-6bc852e96944 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:41.574711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 208/434] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 05:16:41.574711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae263d6-ac4f-4886-bf71-27c85ea5f759 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:41.574711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae263d6-ac4f-4886-bf71-27c85ea5f759 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:41.574711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae263d6-ac4f-4886-bf71-27c85ea5f759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:41.581537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c4333-2fbf-493b-b71e-89637da8978d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:41.582174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c4333-2fbf-493b-b71e-89637da8978d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:41.582455 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c4333-2fbf-493b-b71e-89637da8978d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:41.582727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c4333-2fbf-493b-b71e-89637da8978d None admin] PATH_INFO: `/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:41.583868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c4333-2fbf-493b-b71e-89637da8978d None admin] RBAC: Authorizing `identity:get_project(project_id=0b42b1ccf5fb4ee6b000a7718779b3a3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:41.585333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c4333-2fbf-493b-b71e-89637da8978d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:41.587106 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 227/435] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:41 2022] GET /identity/v3/projects/0b42b1ccf5fb4ee6b000a7718779b3a3 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:41.588879 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c4333-2fbf-493b-b71e-89637da8978d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:41.595381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-1ae263d6-ac4f-4886-bf71-27c85ea5f759 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:41.597048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1ae263d6-ac4f-4886-bf71-27c85ea5f759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:41.625190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:41.667715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 209/436] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:41.668885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae263d6-ac4f-4886-bf71-27c85ea5f759 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:41.704950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c4333-2fbf-493b-b71e-89637da8978d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:41.744672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745883a4-7f30-42b3-8c73-e7b61e4f6fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:41.745326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745883a4-7f30-42b3-8c73-e7b61e4f6fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:41.745788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745883a4-7f30-42b3-8c73-e7b61e4f6fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:41.746135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745883a4-7f30-42b3-8c73-e7b61e4f6fc7 None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:41.747476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745883a4-7f30-42b3-8c73-e7b61e4f6fc7 None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:41.749103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745883a4-7f30-42b3-8c73-e7b61e4f6fc7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:41.751344 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 228/437] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:41 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:41.758258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745883a4-7f30-42b3-8c73-e7b61e4f6fc7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:41.760117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:41.816083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae263d6-ac4f-4886-bf71-27c85ea5f759 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:41.864532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57c4ad-4efd-4456-93cf-fc60dff8ae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:41.865632 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57c4ad-4efd-4456-93cf-fc60dff8ae15 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:41.865922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57c4ad-4efd-4456-93cf-fc60dff8ae15 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:41.866197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57c4ad-4efd-4456-93cf-fc60dff8ae15 None admin] PATH_INFO: `/v3/groups/9d67e7db62f34c61a75e00abbc599884` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:41.867723 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57c4ad-4efd-4456-93cf-fc60dff8ae15 None admin] RBAC: Authorizing `identity:get_group(group_id=9d67e7db62f34c61a75e00abbc599884)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:41.869528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57c4ad-4efd-4456-93cf-fc60dff8ae15 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:41.872242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 210/438] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:41 2022] GET /identity/v3/groups/9d67e7db62f34c61a75e00abbc599884 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:41.877813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57c4ad-4efd-4456-93cf-fc60dff8ae15 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:41.896117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745883a4-7f30-42b3-8c73-e7b61e4f6fc7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:41.899288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:41.933971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348a3ae-c499-44be-a05f-2f23b1cd44a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:41.934601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348a3ae-c499-44be-a05f-2f23b1cd44a3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:41.934886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348a3ae-c499-44be-a05f-2f23b1cd44a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:41.935159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348a3ae-c499-44be-a05f-2f23b1cd44a3 None admin] PATH_INFO: `/v3/groups/9d67e7db62f34c61a75e00abbc599884` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:41.936519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348a3ae-c499-44be-a05f-2f23b1cd44a3 None admin] RBAC: Authorizing `identity:get_group(group_id=9d67e7db62f34c61a75e00abbc599884)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:41.937770 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348a3ae-c499-44be-a05f-2f23b1cd44a3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:41.939515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 229/439] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:41 2022] GET /identity/v3/groups/9d67e7db62f34c61a75e00abbc599884 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:41.940830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a447f30-2a26-4b4e-8610-e5d43cd59b23 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:41.941115 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a447f30-2a26-4b4e-8610-e5d43cd59b23 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:41.941383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a447f30-2a26-4b4e-8610-e5d43cd59b23 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:41.942720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 230/440] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:41.944863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a447f30-2a26-4b4e-8610-e5d43cd59b23 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:41.985688 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57c4ad-4efd-4456-93cf-fc60dff8ae15 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:42.019334 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74b66c-5961-42b2-821e-f9deb31ac62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:42.019892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74b66c-5961-42b2-821e-f9deb31ac62f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:42.020200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74b66c-5961-42b2-821e-f9deb31ac62f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:42.020484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74b66c-5961-42b2-821e-f9deb31ac62f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:42.021208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc74b66c-5961-42b2-821e-f9deb31ac62f None admin] RBAC: Adding query filter params (group.id=9d67e7db62f34c61a75e00abbc599884, role.id=6c6c7fa19ee342da915c90895e707620, scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:42.021518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc74b66c-5961-42b2-821e-f9deb31ac62f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:42.022951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc74b66c-5961-42b2-821e-f9deb31ac62f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:42.025726 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:42.029527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 211/441] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 05:16:41 2022] GET /identity/v3/role_assignments?group.id=9d67e7db62f34c61a75e00abbc599884&scope.project.id=0b42b1ccf5fb4ee6b000a7718779b3a3&role.id=6c6c7fa19ee342da915c90895e707620 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:42.030760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f10584-bca3-448d-92f6-87f34016dd62 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:42.031200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f10584-bca3-448d-92f6-87f34016dd62 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:42.031806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f10584-bca3-448d-92f6-87f34016dd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:42.048159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a447f30-2a26-4b4e-8610-e5d43cd59b23 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:42.052066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a9f10584-bca3-448d-92f6-87f34016dd62 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:42.053303 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a9f10584-bca3-448d-92f6-87f34016dd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:42.086119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d5c7b-ef08-4b02-b924-6d42ad04d6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:42.086746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d5c7b-ef08-4b02-b924-6d42ad04d6bd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:42.087019 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d5c7b-ef08-4b02-b924-6d42ad04d6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:42.087287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d5c7b-ef08-4b02-b924-6d42ad04d6bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:42.088056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d5c7b-ef08-4b02-b924-6d42ad04d6bd None admin] RBAC: Adding query filter params (group.id=9d67e7db62f34c61a75e00abbc599884, role.id=d9f6babaca2d494c9a04f0aca1271af8, scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:42.088364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d5c7b-ef08-4b02-b924-6d42ad04d6bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:42.089840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d5c7b-ef08-4b02-b924-6d42ad04d6bd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:42.096519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 231/442] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 05:16:41 2022] GET /identity/v3/role_assignments?group.id=9d67e7db62f34c61a75e00abbc599884&scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17&role.id=d9f6babaca2d494c9a04f0aca1271af8 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:42.104686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d5c7b-ef08-4b02-b924-6d42ad04d6bd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:42.138704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 212/443] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:42.159043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:42.238933 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d5c7b-ef08-4b02-b924-6d42ad04d6bd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:42.279354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917a63f-ccd1-4169-a844-8804b80a42be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:42.280271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a63f-ccd1-4169-a844-8804b80a42be None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:42.280725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a63f-ccd1-4169-a844-8804b80a42be None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:42.281154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a63f-ccd1-4169-a844-8804b80a42be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:42.293036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917a63f-ccd1-4169-a844-8804b80a42be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-4917a63f-ccd1-4169-a844-8804b80a42be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:42.298381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:42.299267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:42.300792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:42.300792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:42.300792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:42.300792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:42.300792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:42.300792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:42.300792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:42.300792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:42.300792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:42.300792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:42.300792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:42.300792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:42.300792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 232/444] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:16:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:42.305418 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f10584-bca3-448d-92f6-87f34016dd62 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:42.422408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f10584-bca3-448d-92f6-87f34016dd62 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:42.423974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917a63f-ccd1-4169-a844-8804b80a42be None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:42.453244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:42.462021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946685e8-2638-44f5-8698-186af4ece162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:42.462636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946685e8-2638-44f5-8698-186af4ece162 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:42.465704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946685e8-2638-44f5-8698-186af4ece162 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:42.466001 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946685e8-2638-44f5-8698-186af4ece162 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:42.466764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946685e8-2638-44f5-8698-186af4ece162 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:42.466976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946685e8-2638-44f5-8698-186af4ece162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:42.468467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946685e8-2638-44f5-8698-186af4ece162 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:42.479237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 213/445] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:42.488330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946685e8-2638-44f5-8698-186af4ece162 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:42.548383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:42.557785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917a63f-ccd1-4169-a844-8804b80a42be None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:42.600229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a2245-edc1-4d71-8ef2-7be09007e3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:42.600964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a2245-edc1-4d71-8ef2-7be09007e3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:42.601153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a2245-edc1-4d71-8ef2-7be09007e3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:42.601405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a2245-edc1-4d71-8ef2-7be09007e3f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:42.606811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946685e8-2638-44f5-8698-186af4ece162 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:42.614354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a2245-edc1-4d71-8ef2-7be09007e3f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e80a2245-edc1-4d71-8ef2-7be09007e3f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:42.619316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:42.620232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:42.621183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:42.621183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:42.621183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:42.621183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:42.621183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:42.621183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:42.621183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:42.621183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:42.621183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:42.621183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:42.621183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:16:42.621183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:42.621183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 233/446] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:16:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:42.629161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a2245-edc1-4d71-8ef2-7be09007e3f5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:42.650254 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a2492-f7ca-405d-b533-f32257d96721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:42.650872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a2492-f7ca-405d-b533-f32257d96721 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:42.651151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a2492-f7ca-405d-b533-f32257d96721 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:42.651424 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a2492-f7ca-405d-b533-f32257d96721 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:42.711300 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:42.721179 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a2492-f7ca-405d-b533-f32257d96721 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:42.723375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a2492-f7ca-405d-b533-f32257d96721 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:42.732016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a2245-edc1-4d71-8ef2-7be09007e3f5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:42.765958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e11bd-caee-4640-8011-4692c446336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:42.766565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e11bd-caee-4640-8011-4692c446336d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:42.766848 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e11bd-caee-4640-8011-4692c446336d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:42.767123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e11bd-caee-4640-8011-4692c446336d None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:42.767783 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e11bd-caee-4640-8011-4692c446336d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:42.768118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e11bd-caee-4640-8011-4692c446336d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:42.770263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e11bd-caee-4640-8011-4692c446336d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:42.778473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 234/447] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:42.784616 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e11bd-caee-4640-8011-4692c446336d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:42.806392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:42.828952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 214/448] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 05:16:42.835160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a2492-f7ca-405d-b533-f32257d96721 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:42.888560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e11bd-caee-4640-8011-4692c446336d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:42.922081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d24ee-3f2e-4703-99f6-9f0d2768d79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:42.922685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d24ee-3f2e-4703-99f6-9f0d2768d79f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:42.922970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d24ee-3f2e-4703-99f6-9f0d2768d79f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:42.923247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d24ee-3f2e-4703-99f6-9f0d2768d79f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:42.927879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:42.937619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a2492-f7ca-405d-b533-f32257d96721 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:42.970864 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bd628-d46a-401b-8232-d45f366bf2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:42.972025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bd628-d46a-401b-8232-d45f366bf2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:42.972321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bd628-d46a-401b-8232-d45f366bf2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:42.972581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bd628-d46a-401b-8232-d45f366bf2f8 None admin] PATH_INFO: `/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:42.973710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bd628-d46a-401b-8232-d45f366bf2f8 None admin] RBAC: Authorizing `identity:get_project(project_id=7c6f2a4f66134019b9d71c524ab5cb17)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:42.975149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bd628-d46a-401b-8232-d45f366bf2f8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:42.976787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 215/449] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:42 2022] GET /identity/v3/projects/7c6f2a4f66134019b9d71c524ab5cb17 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:42.982319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bd628-d46a-401b-8232-d45f366bf2f8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:42.989186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d24ee-3f2e-4703-99f6-9f0d2768d79f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:42.990646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d24ee-3f2e-4703-99f6-9f0d2768d79f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:43.008738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:43.080501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bd628-d46a-401b-8232-d45f366bf2f8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:43.088283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 235/450] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 05:16:43.093859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d24ee-3f2e-4703-99f6-9f0d2768d79f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:43.114528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf7dfb-f018-43fb-b273-faf66eeae2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:43.115157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf7dfb-f018-43fb-b273-faf66eeae2bf None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:43.115437 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf7dfb-f018-43fb-b273-faf66eeae2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:43.115710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf7dfb-f018-43fb-b273-faf66eeae2bf None admin] PATH_INFO: `/v3/groups/9d67e7db62f34c61a75e00abbc599884` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:43.116853 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf7dfb-f018-43fb-b273-faf66eeae2bf None admin] RBAC: Authorizing `identity:get_group(group_id=9d67e7db62f34c61a75e00abbc599884)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:43.118293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf7dfb-f018-43fb-b273-faf66eeae2bf None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:43.119854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 216/451] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:42 2022] GET /identity/v3/groups/9d67e7db62f34c61a75e00abbc599884 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:43.125660 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf7dfb-f018-43fb-b273-faf66eeae2bf None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:43.154258 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:43.191089 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d24ee-3f2e-4703-99f6-9f0d2768d79f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:43.207464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:43.225087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13926d-fcaa-4d70-bcc5-abe9e78a9907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:43.225352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf7dfb-f018-43fb-b273-faf66eeae2bf None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:43.225715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13926d-fcaa-4d70-bcc5-abe9e78a9907 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:43.225992 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13926d-fcaa-4d70-bcc5-abe9e78a9907 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:43.226261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13926d-fcaa-4d70-bcc5-abe9e78a9907 None admin] PATH_INFO: `/v3/projects/4c05020216314cd9bc11af667152fcc9` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:43.227387 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13926d-fcaa-4d70-bcc5-abe9e78a9907 None admin] RBAC: Authorizing `identity:get_project(project_id=4c05020216314cd9bc11af667152fcc9)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:43.228886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13926d-fcaa-4d70-bcc5-abe9e78a9907 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:43.230503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 236/452] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:16:43 2022] GET /identity/v3/projects/4c05020216314cd9bc11af667152fcc9 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:43.235842 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13926d-fcaa-4d70-bcc5-abe9e78a9907 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:43.260096 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691140b-869c-4702-87bf-976cc09925dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:43.260702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691140b-869c-4702-87bf-976cc09925dd None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:43.260980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691140b-869c-4702-87bf-976cc09925dd None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:43.261252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691140b-869c-4702-87bf-976cc09925dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:43.261975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691140b-869c-4702-87bf-976cc09925dd None admin] RBAC: Adding query filter params (group.id=9d67e7db62f34c61a75e00abbc599884, role.id=6c6c7fa19ee342da915c90895e707620, scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:43.262284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691140b-869c-4702-87bf-976cc09925dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:43.263711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691140b-869c-4702-87bf-976cc09925dd None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:43.270482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 217/453] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 05:16:43 2022] GET /identity/v3/role_assignments?group.id=9d67e7db62f34c61a75e00abbc599884&scope.project.id=7c6f2a4f66134019b9d71c524ab5cb17&role.id=6c6c7fa19ee342da915c90895e707620 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:43.338378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13926d-fcaa-4d70-bcc5-abe9e78a9907 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:43.373683 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2f8ee-737a-44c3-b662-397b500b4460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:43.374325 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2f8ee-737a-44c3-b662-397b500b4460 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:43.374602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2f8ee-737a-44c3-b662-397b500b4460 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:43.374871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2f8ee-737a-44c3-b662-397b500b4460 None admin] PATH_INFO: `/v3/groups/568077d393d240c5b673198e30a3bd45` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:43.376034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2f8ee-737a-44c3-b662-397b500b4460 None admin] RBAC: Authorizing `identity:get_group(group_id=568077d393d240c5b673198e30a3bd45)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:43.377477 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2f8ee-737a-44c3-b662-397b500b4460 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:43.379670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 237/454] 10.222.0.33 () {56 vars in 1180 bytes} [Sun Sep 11 05:16:43 2022] GET /identity/v3/groups/568077d393d240c5b673198e30a3bd45 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:43.385692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691140b-869c-4702-87bf-976cc09925dd None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:43.485493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:43.495408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691140b-869c-4702-87bf-976cc09925dd None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:43.530235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f10632-c9ab-4f6e-8235-8baedc8a65b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:43.530851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10632-c9ab-4f6e-8235-8baedc8a65b6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:43.531135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10632-c9ab-4f6e-8235-8baedc8a65b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:43.531701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10632-c9ab-4f6e-8235-8baedc8a65b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:43.532521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f10632-c9ab-4f6e-8235-8baedc8a65b6 None admin] RBAC: Adding query filter params (group.id=568077d393d240c5b673198e30a3bd45, role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=4c05020216314cd9bc11af667152fcc9) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:43.532830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f10632-c9ab-4f6e-8235-8baedc8a65b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:43.534450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f10632-c9ab-4f6e-8235-8baedc8a65b6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:43.540465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 218/455] 10.222.0.33 () {56 vars in 1376 bytes} [Sun Sep 11 05:16:43 2022] GET /identity/v3/role_assignments?group.id=568077d393d240c5b673198e30a3bd45&scope.project.id=4c05020216314cd9bc11af667152fcc9&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:46.052163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbce156-4e20-402e-bf06-28af5fecf95b None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:46.052706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbce156-4e20-402e-bf06-28af5fecf95b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:46.052706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbce156-4e20-402e-bf06-28af5fecf95b None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:46.054363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 238/456] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:46.060110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75c216-2c7e-49d1-816f-034664ce1825 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:46.060908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75c216-2c7e-49d1-816f-034664ce1825 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:46.061410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75c216-2c7e-49d1-816f-034664ce1825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:46.089084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-be75c216-2c7e-49d1-816f-034664ce1825 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:46.090297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-be75c216-2c7e-49d1-816f-034664ce1825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:46.197520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 219/457] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:46.286360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7cf2d-8bdd-444b-886b-4880782b41cd None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:46.287041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7cf2d-8bdd-444b-886b-4880782b41cd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:46.287521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7cf2d-8bdd-444b-886b-4880782b41cd None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:46.289340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 239/458] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:46.295013 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a1f16-5016-4f60-9c5b-ea200627d6e6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:46.295302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a1f16-5016-4f60-9c5b-ea200627d6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:46.295580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a1f16-5016-4f60-9c5b-ea200627d6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:46.325108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a10a1f16-5016-4f60-9c5b-ea200627d6e6 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:46.326382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a10a1f16-5016-4f60-9c5b-ea200627d6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:46.426963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 220/459] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:46.466392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51896be7-469e-43f9-bdfc-e75ce84a108e None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:46.466899 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51896be7-469e-43f9-bdfc-e75ce84a108e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:46.467348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51896be7-469e-43f9-bdfc-e75ce84a108e None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:46.469105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 240/460] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:46.474161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952d2ef-e8f0-44a0-92e5-8b38c7cd47c1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:46.474447 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952d2ef-e8f0-44a0-92e5-8b38c7cd47c1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:46.474725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952d2ef-e8f0-44a0-92e5-8b38c7cd47c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:46.502564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5952d2ef-e8f0-44a0-92e5-8b38c7cd47c1 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:46.504336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5952d2ef-e8f0-44a0-92e5-8b38c7cd47c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:46.522062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:46.586657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 221/461] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:46.605729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51896be7-469e-43f9-bdfc-e75ce84a108e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:46.686335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:46.743072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51896be7-469e-43f9-bdfc-e75ce84a108e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:46.778551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827e577-3a5f-4764-89bd-7d6dc9769765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:46.779185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827e577-3a5f-4764-89bd-7d6dc9769765 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:46.779466 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827e577-3a5f-4764-89bd-7d6dc9769765 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:46.779741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827e577-3a5f-4764-89bd-7d6dc9769765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:46.785920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952d2ef-e8f0-44a0-92e5-8b38c7cd47c1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:46.797012 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827e577-3a5f-4764-89bd-7d6dc9769765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:46.798631 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827e577-3a5f-4764-89bd-7d6dc9769765 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-2827e577-3a5f-4764-89bd-7d6dc9769765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:46.825123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:46.826093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:46.826975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:46.826975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:46.826975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:46.828199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 241/462] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:46.833293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827e577-3a5f-4764-89bd-7d6dc9769765 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:46.937440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952d2ef-e8f0-44a0-92e5-8b38c7cd47c1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:46.967048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827e577-3a5f-4764-89bd-7d6dc9769765 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:46.976809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:46.982324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc46a0b-fb55-4503-88cd-914939466d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:46.983536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc46a0b-fb55-4503-88cd-914939466d92 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:46.983916 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc46a0b-fb55-4503-88cd-914939466d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:46.984318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc46a0b-fb55-4503-88cd-914939466d92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:46.997898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc46a0b-fb55-4503-88cd-914939466d92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:46.999631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc46a0b-fb55-4503-88cd-914939466d92 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:47.007475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:47.009552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e9e32-5257-4084-93c7-3715040fcfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:47.010775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e9e32-5257-4084-93c7-3715040fcfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:47.011267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e9e32-5257-4084-93c7-3715040fcfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:47.011937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e9e32-5257-4084-93c7-3715040fcfc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:47.012796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e9e32-5257-4084-93c7-3715040fcfc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:47.013249 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e9e32-5257-4084-93c7-3715040fcfc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:47.014888 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e9e32-5257-4084-93c7-3715040fcfc7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-1cc46a0b-fb55-4503-88cd-914939466d92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:47.017938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:47.019106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:47.020223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:47.020223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:47.020223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:47.021569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 222/463] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:47.022451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc46a0b-fb55-4503-88cd-914939466d92 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:47.031269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 242/464] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:47.032438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e9e32-5257-4084-93c7-3715040fcfc7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:47.151667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc46a0b-fb55-4503-88cd-914939466d92 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:47.155827 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e9e32-5257-4084-93c7-3715040fcfc7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:47.198892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fd5b1-935b-4108-8c19-b632b31c62d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:47.199671 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fd5b1-935b-4108-8c19-b632b31c62d2 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:47.200066 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fd5b1-935b-4108-8c19-b632b31c62d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:47.200441 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fd5b1-935b-4108-8c19-b632b31c62d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:47.201005 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8835d-c3ce-4b7d-a872-aa465311c9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:47.201283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fd5b1-935b-4108-8c19-b632b31c62d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:47.201700 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fd5b1-935b-4108-8c19-b632b31c62d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:47.202179 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8835d-c3ce-4b7d-a872-aa465311c9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:47.202606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8835d-c3ce-4b7d-a872-aa465311c9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:47.203029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8835d-c3ce-4b7d-a872-aa465311c9c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:47.203454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fd5b1-935b-4108-8c19-b632b31c62d2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:47.216801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8835d-c3ce-4b7d-a872-aa465311c9c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:47.217209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 243/465] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:47.218162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fd5b1-935b-4108-8c19-b632b31c62d2 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:47.218936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8835d-c3ce-4b7d-a872-aa465311c9c9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-cef8835d-c3ce-4b7d-a872-aa465311c9c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:47.237614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:47.238893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:47.239969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:47.239969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:47.239969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:47.242342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 223/466] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:47.243334 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8835d-c3ce-4b7d-a872-aa465311c9c9 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:47.257790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:47.304313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:47.343934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fd5b1-935b-4108-8c19-b632b31c62d2 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:47.359508 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8835d-c3ce-4b7d-a872-aa465311c9c9 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:47.381008 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7ce82-3bc8-4f8c-848d-0f587539e9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:47.381622 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7ce82-3bc8-4f8c-848d-0f587539e9d2 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:47.381902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7ce82-3bc8-4f8c-848d-0f587539e9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:47.382172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7ce82-3bc8-4f8c-848d-0f587539e9d2 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:47.382931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7ce82-3bc8-4f8c-848d-0f587539e9d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:47.384573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7ce82-3bc8-4f8c-848d-0f587539e9d2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:47.393756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba646042-60db-484a-a42f-a1944c648a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:47.394706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba646042-60db-484a-a42f-a1944c648a15 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:47.395211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba646042-60db-484a-a42f-a1944c648a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:47.395507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba646042-60db-484a-a42f-a1944c648a15 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:47.396172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba646042-60db-484a-a42f-a1944c648a15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:47.398251 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba646042-60db-484a-a42f-a1944c648a15 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:47.409940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 244/467] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:16:47 2022] POST /identity/v3/users => generated 246 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:16:47.411115 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7ce82-3bc8-4f8c-848d-0f587539e9d2 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:47.440515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 224/468] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:16:47 2022] POST /identity/v3/users => generated 248 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:16:47.440515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdde901-ee36-4964-afd2-e4c2905433b6 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:47.440515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdde901-ee36-4964-afd2-e4c2905433b6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:47.440515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdde901-ee36-4964-afd2-e4c2905433b6 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:47.440515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 225/469] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:47.445712 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168232b6-fb0f-420d-b1ae-c4e82e515931 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:47.445982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168232b6-fb0f-420d-b1ae-c4e82e515931 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:47.446274 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168232b6-fb0f-420d-b1ae-c4e82e515931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:47.472116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-168232b6-fb0f-420d-b1ae-c4e82e515931 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:47.472116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-168232b6-fb0f-420d-b1ae-c4e82e515931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:47.624037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 226/470] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:47.625763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:47.648061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7ce82-3bc8-4f8c-848d-0f587539e9d2 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:47.684458 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db039676-440a-4e21-ba4d-85b16da07052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:47.685155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db039676-440a-4e21-ba4d-85b16da07052 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:47.685360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db039676-440a-4e21-ba4d-85b16da07052 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:47.685630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db039676-440a-4e21-ba4d-85b16da07052 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:47.686252 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db039676-440a-4e21-ba4d-85b16da07052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:47.686552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db039676-440a-4e21-ba4d-85b16da07052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:47.687918 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db039676-440a-4e21-ba4d-85b16da07052 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:47.701292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 245/471] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:47.707199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168232b6-fb0f-420d-b1ae-c4e82e515931 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:47.744678 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:47.830229 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168232b6-fb0f-420d-b1ae-c4e82e515931 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:47.871338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16508033-34f1-46be-8fbb-905b2e4ef63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:47.871944 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16508033-34f1-46be-8fbb-905b2e4ef63d None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:47.872281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16508033-34f1-46be-8fbb-905b2e4ef63d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:47.872551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16508033-34f1-46be-8fbb-905b2e4ef63d None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:47.873190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16508033-34f1-46be-8fbb-905b2e4ef63d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:47.874913 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16508033-34f1-46be-8fbb-905b2e4ef63d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:47.881184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4cb5a-6435-4550-ab13-4c4e9959b4ef None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:47.881466 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4cb5a-6435-4550-ab13-4c4e9959b4ef None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:47.881735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4cb5a-6435-4550-ab13-4c4e9959b4ef None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:47.884495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 246/472] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:47.889517 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c03f7a-eb23-4403-a10d-9a91b80ac50c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:47.889799 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c03f7a-eb23-4403-a10d-9a91b80ac50c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:47.890065 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c03f7a-eb23-4403-a10d-9a91b80ac50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:47.902452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 227/473] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:16:47 2022] POST /identity/v3/users => generated 248 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:16:47.912759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-39c03f7a-eb23-4403-a10d-9a91b80ac50c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:47.913995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-39c03f7a-eb23-4403-a10d-9a91b80ac50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:47.941990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16508033-34f1-46be-8fbb-905b2e4ef63d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:48.008202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 247/474] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:48.061794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:48.074535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16508033-34f1-46be-8fbb-905b2e4ef63d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:48.111320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321351bd-9746-4745-bfb4-ba9612cdfa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:48.113025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321351bd-9746-4745-bfb4-ba9612cdfa5b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:48.113514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321351bd-9746-4745-bfb4-ba9612cdfa5b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:48.113981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321351bd-9746-4745-bfb4-ba9612cdfa5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:48.125870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321351bd-9746-4745-bfb4-ba9612cdfa5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:48.127750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321351bd-9746-4745-bfb4-ba9612cdfa5b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-321351bd-9746-4745-bfb4-ba9612cdfa5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:48.145241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:48.146232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:48.147078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:48.147078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:48.147078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:48.149247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 228/475] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:16:48.155900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c03f7a-eb23-4403-a10d-9a91b80ac50c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:48.260973 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:48.299389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c03f7a-eb23-4403-a10d-9a91b80ac50c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:48.311456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321351bd-9746-4745-bfb4-ba9612cdfa5b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:48.341127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf229a-6ba6-48a9-8993-c19cc7d8155f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:48.341748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf229a-6ba6-48a9-8993-c19cc7d8155f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:48.342025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf229a-6ba6-48a9-8993-c19cc7d8155f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:48.342293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf229a-6ba6-48a9-8993-c19cc7d8155f None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:48.342936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf229a-6ba6-48a9-8993-c19cc7d8155f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:48.343235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf229a-6ba6-48a9-8993-c19cc7d8155f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:48.345125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf229a-6ba6-48a9-8993-c19cc7d8155f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:48.358310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 248/476] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:48.371495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf229a-6ba6-48a9-8993-c19cc7d8155f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:48.468342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321351bd-9746-4745-bfb4-ba9612cdfa5b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:48.543077 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44f276-c1b8-4661-ad35-ef3e35fa9a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:48.543605 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44f276-c1b8-4661-ad35-ef3e35fa9a8b None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:48.543879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44f276-c1b8-4661-ad35-ef3e35fa9a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:48.544179 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44f276-c1b8-4661-ad35-ef3e35fa9a8b None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:48.544999 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44f276-c1b8-4661-ad35-ef3e35fa9a8b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:48.546654 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44f276-c1b8-4661-ad35-ef3e35fa9a8b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:48.560994 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.sql.core [None req-ab44f276-c1b8-4661-ad35-ef3e35fa9a8b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 05:16:48.560994 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 11 05:16:48.560994 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [parameters: {'id': '4adf853ef96740958d937ac18a0d3abb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 11 05:16:48.560994 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81445) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 05:16:48.569554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf229a-6ba6-48a9-8993-c19cc7d8155f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ab44f276-c1b8-4661-ad35-ef3e35fa9a8b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application conn.query(q) Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application result.read() Sep 11 05:16:48.593061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return ref.to_dict() Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 05:16:48.594011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application next(self.gen) Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application yield resource Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application next(self.gen) Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise self.value Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application session.commit() Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 05:16:48.594952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.session.flush() Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application compat.raise_( Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 05:16:48.595824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 05:16:48.598707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application conn.query(q) Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application result.read() Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 05:16:48.600759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [parameters: {'id': '4adf853ef96740958d937ac18a0d3abb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:48.602971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 11 05:16:48.605036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:48.608776 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 229/477] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 11 05:16:48 2022] POST /identity/v3/roles => generated 144 bytes in 284 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:16:48.608776 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44f276-c1b8-4661-ad35-ef3e35fa9a8b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:48.619195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:48.629073 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:48.631803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a6c33-cb3c-4f40-8496-7103674ca981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:48.632468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a6c33-cb3c-4f40-8496-7103674ca981 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:48.632755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a6c33-cb3c-4f40-8496-7103674ca981 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:48.633007 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a6c33-cb3c-4f40-8496-7103674ca981 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:48.633775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a6c33-cb3c-4f40-8496-7103674ca981 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:48.635118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a6c33-cb3c-4f40-8496-7103674ca981 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:48.679226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 249/478] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:16:48 2022] POST /identity/v3/users => generated 249 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:16:48.754100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44f276-c1b8-4661-ad35-ef3e35fa9a8b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:48.793484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558fa23-ebf0-4b9f-b9e1-f5f053db83f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:48.794061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558fa23-ebf0-4b9f-b9e1-f5f053db83f8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:48.794382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558fa23-ebf0-4b9f-b9e1-f5f053db83f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:48.794662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558fa23-ebf0-4b9f-b9e1-f5f053db83f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:48.807567 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558fa23-ebf0-4b9f-b9e1-f5f053db83f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-b558fa23-ebf0-4b9f-b9e1-f5f053db83f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:48.813137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:48.814071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:48.814874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:48.814874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:48.814874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:48.814874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:48.814874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:48.814874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:48.814874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:48.814874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:48.814874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:48.814874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:48.814874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 05:16:48.814874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:48.814874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 230/479] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:16:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:48.821045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a6c33-cb3c-4f40-8496-7103674ca981 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:48.968127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a6c33-cb3c-4f40-8496-7103674ca981 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:49.005513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a310cf5-12d9-4543-ac62-f2a222c4ea35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:49.006142 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a310cf5-12d9-4543-ac62-f2a222c4ea35 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:49.006419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a310cf5-12d9-4543-ac62-f2a222c4ea35 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:49.006686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a310cf5-12d9-4543-ac62-f2a222c4ea35 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:49.007367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a310cf5-12d9-4543-ac62-f2a222c4ea35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:49.007667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a310cf5-12d9-4543-ac62-f2a222c4ea35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:49.009135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a310cf5-12d9-4543-ac62-f2a222c4ea35 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:49.024139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 250/480] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:50.058513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74b6de-b99b-4fd7-9633-4bc8344a4c54 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:50.059024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74b6de-b99b-4fd7-9633-4bc8344a4c54 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:50.059024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74b6de-b99b-4fd7-9633-4bc8344a4c54 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:50.060665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 231/481] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:50.065848 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef0b8b-e8cc-4047-b83e-e4a43bcf3654 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:50.066254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef0b8b-e8cc-4047-b83e-e4a43bcf3654 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:50.066446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef0b8b-e8cc-4047-b83e-e4a43bcf3654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:50.089192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-83ef0b8b-e8cc-4047-b83e-e4a43bcf3654 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:50.090392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-83ef0b8b-e8cc-4047-b83e-e4a43bcf3654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:50.153943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:50.180924 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 251/482] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:50.344787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c5657-d9b2-413f-ab27-fbfade35f892 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:50.345140 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c5657-d9b2-413f-ab27-fbfade35f892 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:50.345347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c5657-d9b2-413f-ab27-fbfade35f892 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:50.346810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 232/483] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:50.352157 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0950b4-ebc9-4e3e-87a9-347e160eeb39 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:50.352395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0950b4-ebc9-4e3e-87a9-347e160eeb39 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:50.352675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0950b4-ebc9-4e3e-87a9-347e160eeb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:50.375897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4f0950b4-ebc9-4e3e-87a9-347e160eeb39 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:50.377118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4f0950b4-ebc9-4e3e-87a9-347e160eeb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:50.474088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 252/484] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:50.543105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c5657-d9b2-413f-ab27-fbfade35f892 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:50.617503 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:50.654993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c5657-d9b2-413f-ab27-fbfade35f892 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:50.697237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26817c0-7a9b-4eb5-a972-2a7db51b4c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:50.697925 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26817c0-7a9b-4eb5-a972-2a7db51b4c24 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:50.698213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26817c0-7a9b-4eb5-a972-2a7db51b4c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:50.698484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26817c0-7a9b-4eb5-a972-2a7db51b4c24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:50.714224 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26817c0-7a9b-4eb5-a972-2a7db51b4c24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c26817c0-7a9b-4eb5-a972-2a7db51b4c24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:50.719076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:50.719982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:50.721795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:50.721795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:50.721795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:50.721795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:50.721795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:50.721795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:50.721795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:50.721795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:50.721795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:50.721795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:50.721795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:50.721795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:50.721795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 233/485] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:16:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:50.727005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0950b4-ebc9-4e3e-87a9-347e160eeb39 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:50.852520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0950b4-ebc9-4e3e-87a9-347e160eeb39 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:50.865508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:50.873675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26817c0-7a9b-4eb5-a972-2a7db51b4c24 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:50.908550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06759e-c0cd-4989-9154-abb7330911a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:50.909342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06759e-c0cd-4989-9154-abb7330911a2 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:50.909625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06759e-c0cd-4989-9154-abb7330911a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:50.911054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06759e-c0cd-4989-9154-abb7330911a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:50.911748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06759e-c0cd-4989-9154-abb7330911a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:50.912079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06759e-c0cd-4989-9154-abb7330911a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:50.915342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06759e-c0cd-4989-9154-abb7330911a2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:50.938642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 253/486] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:16:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:50.952521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06759e-c0cd-4989-9154-abb7330911a2 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:51.042431 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26817c0-7a9b-4eb5-a972-2a7db51b4c24 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:51.082022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06759e-c0cd-4989-9154-abb7330911a2 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:51.084358 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:51.088649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92917803-c17f-4765-927e-1dc1ae89123c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:51.089466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92917803-c17f-4765-927e-1dc1ae89123c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:51.089871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92917803-c17f-4765-927e-1dc1ae89123c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:51.090221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92917803-c17f-4765-927e-1dc1ae89123c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:51.105100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92917803-c17f-4765-927e-1dc1ae89123c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-92917803-c17f-4765-927e-1dc1ae89123c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:51.112981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:51.113865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 234/487] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:16:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:16:51.119320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd7766-90c3-48f2-bebd-588aa1233973 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:51.120311 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd7766-90c3-48f2-bebd-588aa1233973 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:51.120311 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd7766-90c3-48f2-bebd-588aa1233973 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:51.120311 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 235/488] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:51.123501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd7766-90c3-48f2-bebd-588aa1233973 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:51.132978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed858f6d-16a4-4f48-b3da-5e840030f250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:51.133938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed858f6d-16a4-4f48-b3da-5e840030f250 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:51.134430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed858f6d-16a4-4f48-b3da-5e840030f250 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:51.134920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed858f6d-16a4-4f48-b3da-5e840030f250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:51.150773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed858f6d-16a4-4f48-b3da-5e840030f250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:51.153137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed858f6d-16a4-4f48-b3da-5e840030f250 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-ed858f6d-16a4-4f48-b3da-5e840030f250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:51.173316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:51.174278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:51.175610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:51.175610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:51.175610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:51.177854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 254/489] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:16:51.179278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cc480-cc3d-4a76-b2da-5936afb97182 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:51.179694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cc480-cc3d-4a76-b2da-5936afb97182 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:51.180114 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cc480-cc3d-4a76-b2da-5936afb97182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:51.203548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-411cc480-cc3d-4a76-b2da-5936afb97182 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:51.205702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-411cc480-cc3d-4a76-b2da-5936afb97182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:51.230081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:51.265659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd7766-90c3-48f2-bebd-588aa1233973 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:51.296140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 255/490] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:51.297215 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cc480-cc3d-4a76-b2da-5936afb97182 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:51.320433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca302d-c4a7-4a75-9f83-6da94e585792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:51.321347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca302d-c4a7-4a75-9f83-6da94e585792 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:51.321768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca302d-c4a7-4a75-9f83-6da94e585792 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:51.322181 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca302d-c4a7-4a75-9f83-6da94e585792 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:51.323039 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca302d-c4a7-4a75-9f83-6da94e585792 None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:51.324044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca302d-c4a7-4a75-9f83-6da94e585792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:51.327433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca302d-c4a7-4a75-9f83-6da94e585792 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:51.344421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 236/491] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:16:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:51.353028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca302d-c4a7-4a75-9f83-6da94e585792 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:51.449220 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:51.490871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cc480-cc3d-4a76-b2da-5936afb97182 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:51.531288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca302d-c4a7-4a75-9f83-6da94e585792 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:51.571664 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f63b6-b801-4a7c-a211-45b840c80772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:51.572389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f63b6-b801-4a7c-a211-45b840c80772 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:51.572667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f63b6-b801-4a7c-a211-45b840c80772 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:51.573377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f63b6-b801-4a7c-a211-45b840c80772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:51.585495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f63b6-b801-4a7c-a211-45b840c80772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:51.587028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f63b6-b801-4a7c-a211-45b840c80772 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-4b9f63b6-b801-4a7c-a211-45b840c80772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:51.603448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:51.604459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:51.605302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:51.605302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:51.605302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:51.609154 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 237/492] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:51.610366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f63b6-b801-4a7c-a211-45b840c80772 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:51.617370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a46c8-2280-4fd3-9658-88343620b858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:51.618005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a46c8-2280-4fd3-9658-88343620b858 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:51.618283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a46c8-2280-4fd3-9658-88343620b858 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:51.618555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a46c8-2280-4fd3-9658-88343620b858 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:51.619215 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a46c8-2280-4fd3-9658-88343620b858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:51.619515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a46c8-2280-4fd3-9658-88343620b858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:51.620935 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a46c8-2280-4fd3-9658-88343620b858 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:51.634164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 256/493] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:51.635265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee26cc-e5b4-4080-998b-66da4eb76cd2 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:51.635551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee26cc-e5b4-4080-998b-66da4eb76cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:51.635826 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee26cc-e5b4-4080-998b-66da4eb76cd2 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:51.637656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 257/494] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:51.638383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee26cc-e5b4-4080-998b-66da4eb76cd2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:51.660079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:51.727875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f63b6-b801-4a7c-a211-45b840c80772 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:51.729843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:51.761377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee26cc-e5b4-4080-998b-66da4eb76cd2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:51.769017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e97e9-6cf6-4bdd-8497-af9db874a587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:51.769728 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e97e9-6cf6-4bdd-8497-af9db874a587 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:51.770009 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e97e9-6cf6-4bdd-8497-af9db874a587 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:51.770286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e97e9-6cf6-4bdd-8497-af9db874a587 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:51.771122 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e97e9-6cf6-4bdd-8497-af9db874a587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:51.771432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e97e9-6cf6-4bdd-8497-af9db874a587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:51.772981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e97e9-6cf6-4bdd-8497-af9db874a587 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:51.794496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 238/495] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:51.795736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444d42f-b32b-4936-b276-64f6d2bff304 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:51.796041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444d42f-b32b-4936-b276-64f6d2bff304 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:51.796353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444d42f-b32b-4936-b276-64f6d2bff304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:51.800935 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19e223-03f5-4fd7-b9fa-81c48f3d1fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:51.801577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19e223-03f5-4fd7-b9fa-81c48f3d1fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:51.801857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19e223-03f5-4fd7-b9fa-81c48f3d1fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:51.802130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19e223-03f5-4fd7-b9fa-81c48f3d1fd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:51.814865 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19e223-03f5-4fd7-b9fa-81c48f3d1fd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:51.816146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8444d42f-b32b-4936-b276-64f6d2bff304 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:51.817563 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8444d42f-b32b-4936-b276-64f6d2bff304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-ab19e223-03f5-4fd7-b9fa-81c48f3d1fd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:51.821881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:51.822747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:51.823549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:51.823549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:51.823549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:51.823549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:51.823549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:51.823549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:51.823549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:51.823549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:51.823549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:51.823549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:51.823549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:51.823549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:51.827642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 258/496] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:16:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:51.828691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19e223-03f5-4fd7-b9fa-81c48f3d1fd8 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:51.889360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 239/497] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:51.892421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444d42f-b32b-4936-b276-64f6d2bff304 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:51.907448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:51.934929 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19e223-03f5-4fd7-b9fa-81c48f3d1fd8 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:51.969650 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad314ec-57db-4f90-9ed0-7fcafdecf921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:51.970268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad314ec-57db-4f90-9ed0-7fcafdecf921 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:51.970546 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad314ec-57db-4f90-9ed0-7fcafdecf921 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:51.970817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad314ec-57db-4f90-9ed0-7fcafdecf921 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:51.979407 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad314ec-57db-4f90-9ed0-7fcafdecf921 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:51.980812 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad314ec-57db-4f90-9ed0-7fcafdecf921 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:51.984899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-0ad314ec-57db-4f90-9ed0-7fcafdecf921 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:16:51.993767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:51.994665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:52.000221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:52.000221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:52.000221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:16:52.000221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:16:52.000221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:16:52.000221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:16:52.000221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:16:52.000221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:16:52.000221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 05:16:52.000221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:52.000760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444d42f-b32b-4936-b276-64f6d2bff304 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:52.000832 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 259/498] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:52.000832 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad314ec-57db-4f90-9ed0-7fcafdecf921 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:52.037665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052720a0-58c9-4c29-b918-72f8ba1510ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:52.038409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052720a0-58c9-4c29-b918-72f8ba1510ad None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:52.038760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052720a0-58c9-4c29-b918-72f8ba1510ad None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:52.039104 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052720a0-58c9-4c29-b918-72f8ba1510ad None admin] PATH_INFO: `/v3/users/nova` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:52.046225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052720a0-58c9-4c29-b918-72f8ba1510ad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:52.047675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052720a0-58c9-4c29-b918-72f8ba1510ad None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-052720a0-58c9-4c29-b918-72f8ba1510ad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:16:52.060560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:52.066463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:52.067414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:52.067414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:52.067414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:16:52.067414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:16:52.067414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:16:52.067414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:16:52.067414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:16:52.067414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:16:52.067414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:16:52.067414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:52.067414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 240/499] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 05:16:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:52.067414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f33cf-c53d-43d7-a937-4c733ac38a9c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:52.067414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f33cf-c53d-43d7-a937-4c733ac38a9c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:52.068108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f33cf-c53d-43d7-a937-4c733ac38a9c None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:52.069478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 241/500] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:52.070218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f33cf-c53d-43d7-a937-4c733ac38a9c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:52.110072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad314ec-57db-4f90-9ed0-7fcafdecf921 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:52.145756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab7af2-f27f-4c8b-8e52-45ef70d3d285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:52.146467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab7af2-f27f-4c8b-8e52-45ef70d3d285 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:52.146748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab7af2-f27f-4c8b-8e52-45ef70d3d285 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:52.147016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab7af2-f27f-4c8b-8e52-45ef70d3d285 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:52.147665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab7af2-f27f-4c8b-8e52-45ef70d3d285 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:52.147963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab7af2-f27f-4c8b-8e52-45ef70d3d285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:52.149356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab7af2-f27f-4c8b-8e52-45ef70d3d285 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:52.150799 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:52.159294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 260/501] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:16:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:52.160933 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28886a-a6c7-40e3-8ec1-016490fbdf07 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:52.161218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28886a-a6c7-40e3-8ec1-016490fbdf07 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:52.161485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28886a-a6c7-40e3-8ec1-016490fbdf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:52.179244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fa28886a-a6c7-40e3-8ec1-016490fbdf07 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:52.180467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fa28886a-a6c7-40e3-8ec1-016490fbdf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:52.185804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f33cf-c53d-43d7-a937-4c733ac38a9c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:52.219742 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9063d91-4b52-439b-96df-789553a0991b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:52.220369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9063d91-4b52-439b-96df-789553a0991b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:52.220651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9063d91-4b52-439b-96df-789553a0991b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:52.220921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9063d91-4b52-439b-96df-789553a0991b None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:52.221973 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9063d91-4b52-439b-96df-789553a0991b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:52.222280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9063d91-4b52-439b-96df-789553a0991b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:52.223727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9063d91-4b52-439b-96df-789553a0991b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:52.246085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 261/502] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:52.246843 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28886a-a6c7-40e3-8ec1-016490fbdf07 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:52.269951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 242/503] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 11 05:16:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:52.271150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9063d91-4b52-439b-96df-789553a0991b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:52.283395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:52.357761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28886a-a6c7-40e3-8ec1-016490fbdf07 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:52.378353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9063d91-4b52-439b-96df-789553a0991b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:52.386980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:52.398033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33154b-0632-4726-b8d1-99d017c0d20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:52.398668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33154b-0632-4726-b8d1-99d017c0d20b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:52.398944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33154b-0632-4726-b8d1-99d017c0d20b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:52.399219 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33154b-0632-4726-b8d1-99d017c0d20b None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:52.399945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33154b-0632-4726-b8d1-99d017c0d20b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:52.400283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33154b-0632-4726-b8d1-99d017c0d20b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:52.401706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33154b-0632-4726-b8d1-99d017c0d20b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:52.413837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c9fdf-c22b-4bbc-b8f8-341f75061a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:52.414733 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c9fdf-c22b-4bbc-b8f8-341f75061a64 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:52.415209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c9fdf-c22b-4bbc-b8f8-341f75061a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:52.415674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c9fdf-c22b-4bbc-b8f8-341f75061a64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:52.426910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c9fdf-c22b-4bbc-b8f8-341f75061a64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:52.428505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c9fdf-c22b-4bbc-b8f8-341f75061a64 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:52.438026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 262/504] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 05:16:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:52.438851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33154b-0632-4726-b8d1-99d017c0d20b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-1d7c9fdf-c22b-4bbc-b8f8-341f75061a64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:52.448203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:52.449188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:52.450231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:52.450231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:52.450231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:52.451842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 243/505] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:52.452828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c9fdf-c22b-4bbc-b8f8-341f75061a64 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:52.517048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:52.548135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33154b-0632-4726-b8d1-99d017c0d20b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:52.556093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c9fdf-c22b-4bbc-b8f8-341f75061a64 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:52.583463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db7a9e-5e03-49fa-a817-8973e2c3ece1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:52.584328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db7a9e-5e03-49fa-a817-8973e2c3ece1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:52.584616 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db7a9e-5e03-49fa-a817-8973e2c3ece1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:52.584891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db7a9e-5e03-49fa-a817-8973e2c3ece1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:52.589095 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696ed92-97a4-42df-9f6d-7cde8668ce79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:52.589703 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696ed92-97a4-42df-9f6d-7cde8668ce79 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:52.589986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696ed92-97a4-42df-9f6d-7cde8668ce79 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:52.590259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696ed92-97a4-42df-9f6d-7cde8668ce79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:52.594694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db7a9e-5e03-49fa-a817-8973e2c3ece1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:52.596254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db7a9e-5e03-49fa-a817-8973e2c3ece1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-59db7a9e-5e03-49fa-a817-8973e2c3ece1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:52.610727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:52.611628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:52.613175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:52.613175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:52.613175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:52.613175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 263/506] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:52.613175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db7a9e-5e03-49fa-a817-8973e2c3ece1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:52.648945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:52.662677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696ed92-97a4-42df-9f6d-7cde8668ce79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:52.664295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696ed92-97a4-42df-9f6d-7cde8668ce79 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:52.714022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db7a9e-5e03-49fa-a817-8973e2c3ece1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:52.723743 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:52.748911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e19da-9ebb-4ce6-b594-a68f890ac9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:52.749512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e19da-9ebb-4ce6-b594-a68f890ac9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:52.749794 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e19da-9ebb-4ce6-b594-a68f890ac9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:52.750067 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e19da-9ebb-4ce6-b594-a68f890ac9d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:52.763917 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 244/507] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:52.764852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696ed92-97a4-42df-9f6d-7cde8668ce79 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:52.820313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e19da-9ebb-4ce6-b594-a68f890ac9d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:52.821779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e19da-9ebb-4ce6-b594-a68f890ac9d5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:52.854299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:52.869690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696ed92-97a4-42df-9f6d-7cde8668ce79 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:52.908387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af1455-4415-43e2-8aa8-9f356b97da35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:52.909767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af1455-4415-43e2-8aa8-9f356b97da35 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:52.910042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af1455-4415-43e2-8aa8-9f356b97da35 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:52.910318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af1455-4415-43e2-8aa8-9f356b97da35 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:52.910961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af1455-4415-43e2-8aa8-9f356b97da35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:52.911265 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af1455-4415-43e2-8aa8-9f356b97da35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:52.912700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af1455-4415-43e2-8aa8-9f356b97da35 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:52.922338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 264/508] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:52.923167 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e19da-9ebb-4ce6-b594-a68f890ac9d5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:52.927488 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 245/509] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:52.927488 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af1455-4415-43e2-8aa8-9f356b97da35 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:52.952085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:53.027306 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e19da-9ebb-4ce6-b594-a68f890ac9d5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:53.028592 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af1455-4415-43e2-8aa8-9f356b97da35 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:53.061631 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b9fdd-4c18-4bc5-bf8a-bff609ff1c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:53.062176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b9fdd-4c18-4bc5-bf8a-bff609ff1c8a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:53.062455 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b9fdd-4c18-4bc5-bf8a-bff609ff1c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:53.062727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b9fdd-4c18-4bc5-bf8a-bff609ff1c8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:53.063068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16170c-1e6b-4f05-999d-226869a70a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:53.063668 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16170c-1e6b-4f05-999d-226869a70a26 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:53.063952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16170c-1e6b-4f05-999d-226869a70a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:53.064255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16170c-1e6b-4f05-999d-226869a70a26 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:53.064974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16170c-1e6b-4f05-999d-226869a70a26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:53.065283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16170c-1e6b-4f05-999d-226869a70a26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:53.065581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:53.066712 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16170c-1e6b-4f05-999d-226869a70a26 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:53.074050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b9fdd-4c18-4bc5-bf8a-bff609ff1c8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:53.078343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 246/510] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-679b9fdd-4c18-4bc5-bf8a-bff609ff1c8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:53.078911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:53.080130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:53.081617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:53.081617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:53.081617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:53.081617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:53.081617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:53.081617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:53.081617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:53.081617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:53.081617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:53.081617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:53.081617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:53.081617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:53.082369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16170c-1e6b-4f05-999d-226869a70a26 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:53.082445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 265/511] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:16:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:53.082445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b9fdd-4c18-4bc5-bf8a-bff609ff1c8a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:53.166500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:53.183075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b9fdd-4c18-4bc5-bf8a-bff609ff1c8a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:53.185737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16170c-1e6b-4f05-999d-226869a70a26 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:53.215972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66236f-8b17-4e29-b46e-3ee94ca80c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:53.216617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66236f-8b17-4e29-b46e-3ee94ca80c55 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:53.216897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66236f-8b17-4e29-b46e-3ee94ca80c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:53.217166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66236f-8b17-4e29-b46e-3ee94ca80c55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:53.219492 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005de31-64b2-4495-9cbb-653aac321bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:53.220136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005de31-64b2-4495-9cbb-653aac321bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:53.220413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005de31-64b2-4495-9cbb-653aac321bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:53.220681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005de31-64b2-4495-9cbb-653aac321bd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:53.227111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66236f-8b17-4e29-b46e-3ee94ca80c55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:53.228602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66236f-8b17-4e29-b46e-3ee94ca80c55 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:53.230692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005de31-64b2-4495-9cbb-653aac321bd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:53.232146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005de31-64b2-4495-9cbb-653aac321bd3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-1c66236f-8b17-4e29-b46e-3ee94ca80c55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:53.243321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:53.244275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:53.245130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:53.245130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:53.245130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:53.245130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 266/512] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:53.245345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66236f-8b17-4e29-b46e-3ee94ca80c55 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-7005de31-64b2-4495-9cbb-653aac321bd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:53.246919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:53.248026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:53.248899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:53.248899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:53.248899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:53.248899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 247/513] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:53.249447 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005de31-64b2-4495-9cbb-653aac321bd3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:53.298004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:53.349183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66236f-8b17-4e29-b46e-3ee94ca80c55 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:53.349649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005de31-64b2-4495-9cbb-653aac321bd3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:53.382885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39215cf-da76-414d-94cd-a50873d9f79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:53.383480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39215cf-da76-414d-94cd-a50873d9f79e None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:53.383579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d8c0d-34b7-4bde-82d6-ad8d739c031a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:53.383757 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39215cf-da76-414d-94cd-a50873d9f79e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:53.384047 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39215cf-da76-414d-94cd-a50873d9f79e None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:53.384142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8c0d-34b7-4bde-82d6-ad8d739c031a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:53.384423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8c0d-34b7-4bde-82d6-ad8d739c031a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:53.384677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39215cf-da76-414d-94cd-a50873d9f79e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:53.384766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8c0d-34b7-4bde-82d6-ad8d739c031a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:53.386084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39215cf-da76-414d-94cd-a50873d9f79e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:53.386567 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:53.394153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d8c0d-34b7-4bde-82d6-ad8d739c031a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:53.395601 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d8c0d-34b7-4bde-82d6-ad8d739c031a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:53.407284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 267/514] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:16:53 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-968d8c0d-34b7-4bde-82d6-ad8d739c031a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:16:53.407690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:53.408694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:53.409626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:53.409626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:53.409626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:16:53.409626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:16:53.409626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:16:53.409626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:16:53.409626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:16:53.409626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:16:53.409626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 05:16:53.409626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:53.410165 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39215cf-da76-414d-94cd-a50873d9f79e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:53.410235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 248/515] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:53.410235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d8c0d-34b7-4bde-82d6-ad8d739c031a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:53.530038 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39215cf-da76-414d-94cd-a50873d9f79e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:53.532607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:53.535094 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d8c0d-34b7-4bde-82d6-ad8d739c031a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:53.575021 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd48394-10b7-4741-a04e-72ed0965969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:53.575633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd48394-10b7-4741-a04e-72ed0965969d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:53.575737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de6399-c776-4387-ac66-86dea80efe17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:53.575945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd48394-10b7-4741-a04e-72ed0965969d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:53.576207 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd48394-10b7-4741-a04e-72ed0965969d None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:53.576364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de6399-c776-4387-ac66-86dea80efe17 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:53.576647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de6399-c776-4387-ac66-86dea80efe17 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:53.577616 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd48394-10b7-4741-a04e-72ed0965969d None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:53.577719 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de6399-c776-4387-ac66-86dea80efe17 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:53.577790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd48394-10b7-4741-a04e-72ed0965969d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:53.577857 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de6399-c776-4387-ac66-86dea80efe17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:53.577857 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de6399-c776-4387-ac66-86dea80efe17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:53.578534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd48394-10b7-4741-a04e-72ed0965969d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:53.579185 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de6399-c776-4387-ac66-86dea80efe17 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:53.588298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 268/516] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:16:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:53.588819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd48394-10b7-4741-a04e-72ed0965969d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:53.591945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 249/517] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:53.596697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de6399-c776-4387-ac66-86dea80efe17 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:53.666057 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:53.707704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd48394-10b7-4741-a04e-72ed0965969d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:53.720258 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de6399-c776-4387-ac66-86dea80efe17 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:53.744591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c15796-d256-4ddb-9031-ab7d85f4693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:53.745196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c15796-d256-4ddb-9031-ab7d85f4693a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:53.745531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c15796-d256-4ddb-9031-ab7d85f4693a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:53.745813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c15796-d256-4ddb-9031-ab7d85f4693a None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:53.746637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c15796-d256-4ddb-9031-ab7d85f4693a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:53.747267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c15796-d256-4ddb-9031-ab7d85f4693a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:53.748656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c15796-d256-4ddb-9031-ab7d85f4693a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:53.758139 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2e7c9-d0f8-42ec-ae43-449312d88dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:53.758746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2e7c9-d0f8-42ec-ae43-449312d88dac None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:53.759026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2e7c9-d0f8-42ec-ae43-449312d88dac None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:53.759298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2e7c9-d0f8-42ec-ae43-449312d88dac None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:53.760040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2e7c9-d0f8-42ec-ae43-449312d88dac None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:53.760346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2e7c9-d0f8-42ec-ae43-449312d88dac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:53.765839 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 269/518] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:53.765975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2e7c9-d0f8-42ec-ae43-449312d88dac None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:53.766051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c15796-d256-4ddb-9031-ab7d85f4693a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:53.786450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 250/519] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 11 05:16:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:53.786450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2e7c9-d0f8-42ec-ae43-449312d88dac None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:53.807322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:53.884920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c15796-d256-4ddb-9031-ab7d85f4693a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:53.890710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2e7c9-d0f8-42ec-ae43-449312d88dac None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:53.906197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:53.922259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec63202-bf14-458b-b9f3-0c2142c3df24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:53.922428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec63202-bf14-458b-b9f3-0c2142c3df24 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:53.922688 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec63202-bf14-458b-b9f3-0c2142c3df24 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:53.922950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec63202-bf14-458b-b9f3-0c2142c3df24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:53.930231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416951fa-d726-4dee-aeb7-bb582fabc378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:53.930868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416951fa-d726-4dee-aeb7-bb582fabc378 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:53.931151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416951fa-d726-4dee-aeb7-bb582fabc378 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:53.931425 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416951fa-d726-4dee-aeb7-bb582fabc378 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:53.936760 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec63202-bf14-458b-b9f3-0c2142c3df24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:53.938211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec63202-bf14-458b-b9f3-0c2142c3df24 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:53.938430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416951fa-d726-4dee-aeb7-bb582fabc378 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:53.939913 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416951fa-d726-4dee-aeb7-bb582fabc378 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-416951fa-d726-4dee-aeb7-bb582fabc378 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:16:53.951912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:53.952976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:53.953832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:16:53.953832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:16:53.953832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:16:53.953832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:16:53.953832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:16:53.953832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:16:53.953832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:16:53.953832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:53.953832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 251/520] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:53.953832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416951fa-d726-4dee-aeb7-bb582fabc378 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-eec63202-bf14-458b-b9f3-0c2142c3df24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:53.956208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:53.957090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:53.957948 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:53.957948 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:53.957948 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:53.964526 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 270/521] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:53.965428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec63202-bf14-458b-b9f3-0c2142c3df24 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:54.047081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:54.063068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416951fa-d726-4dee-aeb7-bb582fabc378 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:54.070914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec63202-bf14-458b-b9f3-0c2142c3df24 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:54.100182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d281c-9656-4e35-893b-d731d4852737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:54.100823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d281c-9656-4e35-893b-d731d4852737 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:54.101108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d281c-9656-4e35-893b-d731d4852737 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:54.101377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d281c-9656-4e35-893b-d731d4852737 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:54.105815 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea35d0-5f8e-4f5f-b3eb-152c0cb7d0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:54.106419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea35d0-5f8e-4f5f-b3eb-152c0cb7d0ab None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:54.106698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea35d0-5f8e-4f5f-b3eb-152c0cb7d0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:54.106968 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea35d0-5f8e-4f5f-b3eb-152c0cb7d0ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:54.108257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d281c-9656-4e35-893b-d731d4852737 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:54.109715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d281c-9656-4e35-893b-d731d4852737 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-303d281c-9656-4e35-893b-d731d4852737 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:16:54.121920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:54.122910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:54.123751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:16:54.123751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:16:54.123751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:16:54.123751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:16:54.123751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:16:54.123751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:16:54.123751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:16:54.123751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:54.123751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 252/522] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:54.124776 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d281c-9656-4e35-893b-d731d4852737 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:54.157964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:54.186984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea35d0-5f8e-4f5f-b3eb-152c0cb7d0ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:54.188828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea35d0-5f8e-4f5f-b3eb-152c0cb7d0ab None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:54.245909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d281c-9656-4e35-893b-d731d4852737 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:54.272861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:54.283001 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cab06-2f1c-40ea-86a7-d119447809a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:54.284148 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cab06-2f1c-40ea-86a7-d119447809a4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:54.284665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cab06-2f1c-40ea-86a7-d119447809a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:54.284866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cab06-2f1c-40ea-86a7-d119447809a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:54.285569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cab06-2f1c-40ea-86a7-d119447809a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:54.285878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cab06-2f1c-40ea-86a7-d119447809a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:54.287333 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cab06-2f1c-40ea-86a7-d119447809a4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:54.296418 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 271/523] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:54.301068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea35d0-5f8e-4f5f-b3eb-152c0cb7d0ab None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:54.306113 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 253/524] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:16:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:54.306763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cab06-2f1c-40ea-86a7-d119447809a4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:54.411381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:54.417678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea35d0-5f8e-4f5f-b3eb-152c0cb7d0ab None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:54.422407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cab06-2f1c-40ea-86a7-d119447809a4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:54.454242 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1ff3d-3604-40a8-8db6-27c5bb7fad20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:54.454769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1ff3d-3604-40a8-8db6-27c5bb7fad20 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:54.455070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1ff3d-3604-40a8-8db6-27c5bb7fad20 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:54.455333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1ff3d-3604-40a8-8db6-27c5bb7fad20 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:54.456540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1ff3d-3604-40a8-8db6-27c5bb7fad20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:54.456836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1ff3d-3604-40a8-8db6-27c5bb7fad20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:54.458199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1ff3d-3604-40a8-8db6-27c5bb7fad20 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:54.459521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b039f65-55be-4e2a-8b25-4570dc02d11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:54.460149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b039f65-55be-4e2a-8b25-4570dc02d11a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:54.460432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b039f65-55be-4e2a-8b25-4570dc02d11a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:54.460708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b039f65-55be-4e2a-8b25-4570dc02d11a None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:54.461399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b039f65-55be-4e2a-8b25-4570dc02d11a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:54.461707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b039f65-55be-4e2a-8b25-4570dc02d11a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:54.463143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b039f65-55be-4e2a-8b25-4570dc02d11a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:54.471131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 272/525] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:54.472247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1ff3d-3604-40a8-8db6-27c5bb7fad20 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:54.480040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 254/526] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:16:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:54.484813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b039f65-55be-4e2a-8b25-4570dc02d11a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:54.518980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:54.581080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1ff3d-3604-40a8-8db6-27c5bb7fad20 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:54.588551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b039f65-55be-4e2a-8b25-4570dc02d11a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:54.614963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee539e-9976-48bc-835d-a951dfb4577f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:54.615596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee539e-9976-48bc-835d-a951dfb4577f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:54.615874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee539e-9976-48bc-835d-a951dfb4577f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:54.616179 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee539e-9976-48bc-835d-a951dfb4577f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:54.623942 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b427226-ae90-4454-867b-586096975e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:54.624891 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b427226-ae90-4454-867b-586096975e6e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:54.625345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b427226-ae90-4454-867b-586096975e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:54.625798 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b427226-ae90-4454-867b-586096975e6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:54.626483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee539e-9976-48bc-835d-a951dfb4577f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:54.626909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b427226-ae90-4454-867b-586096975e6e None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=3bd5a9a09f034c05815c631a44d1cdd9) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:54.627457 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b427226-ae90-4454-867b-586096975e6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:54.628183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee539e-9976-48bc-835d-a951dfb4577f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:54.629754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b427226-ae90-4454-867b-586096975e6e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:54.635597 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 255/527] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:54 2022] GET /identity/v3/role_assignments?user.id=3bd5a9a09f034c05815c631a44d1cdd9&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:54.636727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b427226-ae90-4454-867b-586096975e6e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-faee539e-9976-48bc-835d-a951dfb4577f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:54.651844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:54.652951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:54.653790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:54.653790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:54.653790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:54.655375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 273/528] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:54.656503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee539e-9976-48bc-835d-a951dfb4577f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:54.666670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:54.771255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:54.778749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b427226-ae90-4454-867b-586096975e6e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:54.782989 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee539e-9976-48bc-835d-a951dfb4577f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:54.831628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db21b9-4231-4393-b44b-d66ce6cc314c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:54.832294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db21b9-4231-4393-b44b-d66ce6cc314c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:54.832666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db21b9-4231-4393-b44b-d66ce6cc314c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:54.832890 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db21b9-4231-4393-b44b-d66ce6cc314c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:54.833673 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db21b9-4231-4393-b44b-d66ce6cc314c None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=0330c79c391d463b91f99e030c6dcc07) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:54.833978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db21b9-4231-4393-b44b-d66ce6cc314c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:54.835431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db21b9-4231-4393-b44b-d66ce6cc314c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:54.836668 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3377eb7-b323-422e-8ced-c10c5537ac26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:54.837957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3377eb7-b323-422e-8ced-c10c5537ac26 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:54.838427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3377eb7-b323-422e-8ced-c10c5537ac26 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:54.839527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3377eb7-b323-422e-8ced-c10c5537ac26 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:54.841798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 274/529] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:54 2022] GET /identity/v3/role_assignments?user.id=0330c79c391d463b91f99e030c6dcc07&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:54.848971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db21b9-4231-4393-b44b-d66ce6cc314c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:54.850121 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3377eb7-b323-422e-8ced-c10c5537ac26 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:54.851800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3377eb7-b323-422e-8ced-c10c5537ac26 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c3377eb7-b323-422e-8ced-c10c5537ac26 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:16:54.868448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:54.869453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:54.870369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:54.870369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:54.870369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:16:54.870369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:16:54.870369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:16:54.870369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:16:54.870369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:16:54.870369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:16:54.870369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 05:16:54.870369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:54.873578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 256/530] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:16:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:54.879896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3377eb7-b323-422e-8ced-c10c5537ac26 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:54.985215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:55.018126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3377eb7-b323-422e-8ced-c10c5537ac26 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:55.059382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db21b9-4231-4393-b44b-d66ce6cc314c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:55.064265 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd496d-7946-46f4-ad91-dc8050f417ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:55.065894 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd496d-7946-46f4-ad91-dc8050f417ad None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:55.066411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd496d-7946-46f4-ad91-dc8050f417ad None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:55.066936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd496d-7946-46f4-ad91-dc8050f417ad None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:55.068455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd496d-7946-46f4-ad91-dc8050f417ad None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:55.068980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd496d-7946-46f4-ad91-dc8050f417ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:55.070722 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd496d-7946-46f4-ad91-dc8050f417ad None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:55.095864 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 257/531] 10.222.0.33 () {56 vars in 1139 bytes} [Sun Sep 11 05:16:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:55.104820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd496d-7946-46f4-ad91-dc8050f417ad None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:55.108097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82b65b-4ff6-4aef-8dba-48604aff38a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:55.110025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82b65b-4ff6-4aef-8dba-48604aff38a6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:55.110695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82b65b-4ff6-4aef-8dba-48604aff38a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:55.120473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82b65b-4ff6-4aef-8dba-48604aff38a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:55.121126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82b65b-4ff6-4aef-8dba-48604aff38a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:55.121439 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82b65b-4ff6-4aef-8dba-48604aff38a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:55.122839 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82b65b-4ff6-4aef-8dba-48604aff38a6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:55.150472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 275/532] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:55.155856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82b65b-4ff6-4aef-8dba-48604aff38a6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:55.195746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:55.226482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd496d-7946-46f4-ad91-dc8050f417ad None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:55.265632 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4c526-fc0d-4309-92ce-f2256849f6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:55.267041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4c526-fc0d-4309-92ce-f2256849f6bd None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:55.267549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4c526-fc0d-4309-92ce-f2256849f6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:55.268098 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4c526-fc0d-4309-92ce-f2256849f6bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:55.271293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:55.348931 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4c526-fc0d-4309-92ce-f2256849f6bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:55.350914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4c526-fc0d-4309-92ce-f2256849f6bd None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:55.363035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82b65b-4ff6-4aef-8dba-48604aff38a6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:55.429328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db93ed-24ea-4d6a-81b9-c9fa3aa959e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:55.429976 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db93ed-24ea-4d6a-81b9-c9fa3aa959e1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:55.430256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db93ed-24ea-4d6a-81b9-c9fa3aa959e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:55.431315 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db93ed-24ea-4d6a-81b9-c9fa3aa959e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:55.443281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db93ed-24ea-4d6a-81b9-c9fa3aa959e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:55.444770 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db93ed-24ea-4d6a-81b9-c9fa3aa959e1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:55.466062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 258/533] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-25db93ed-24ea-4d6a-81b9-c9fa3aa959e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:16:55.472050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:55.472932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:55.473792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:16:55.473792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:16:55.473792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:16:55.473792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:16:55.473792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:16:55.473792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:16:55.473792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:16:55.473792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:55.473792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 276/534] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:16:55.475786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4c526-fc0d-4309-92ce-f2256849f6bd None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:55.479712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db93ed-24ea-4d6a-81b9-c9fa3aa959e1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:55.542532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:55.608475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:55.634769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db93ed-24ea-4d6a-81b9-c9fa3aa959e1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:55.682234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ceb35-7d4b-4e8b-8f94-acd30353d047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:55.682981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ceb35-7d4b-4e8b-8f94-acd30353d047 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:55.683264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ceb35-7d4b-4e8b-8f94-acd30353d047 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:55.683537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ceb35-7d4b-4e8b-8f94-acd30353d047 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:55.684278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ceb35-7d4b-4e8b-8f94-acd30353d047 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:55.684583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ceb35-7d4b-4e8b-8f94-acd30353d047 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:55.686301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ceb35-7d4b-4e8b-8f94-acd30353d047 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:55.689993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4c526-fc0d-4309-92ce-f2256849f6bd None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:55.706868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 277/535] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:16:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:55.708265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4a755-7297-434f-a591-f3156b071c50 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:55.708548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4a755-7297-434f-a591-f3156b071c50 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:55.708824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4a755-7297-434f-a591-f3156b071c50 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:55.714506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 278/536] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:55.715294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4a755-7297-434f-a591-f3156b071c50 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:55.740864 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72515701-ebf4-4548-a396-ae3716695434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:55.741657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72515701-ebf4-4548-a396-ae3716695434 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:55.742036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72515701-ebf4-4548-a396-ae3716695434 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:55.742408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72515701-ebf4-4548-a396-ae3716695434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:55.757484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72515701-ebf4-4548-a396-ae3716695434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:55.758950 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72515701-ebf4-4548-a396-ae3716695434 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-72515701-ebf4-4548-a396-ae3716695434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:55.782023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:55.782979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:55.783899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:55.783899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:55.783899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:55.786133 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 259/537] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:55.787642 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94413a-53fa-4591-8104-c035d9cc19d6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:55.788175 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94413a-53fa-4591-8104-c035d9cc19d6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:55.791382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94413a-53fa-4591-8104-c035d9cc19d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:55.815364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0f94413a-53fa-4591-8104-c035d9cc19d6 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:55.817066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0f94413a-53fa-4591-8104-c035d9cc19d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:55.838894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4a755-7297-434f-a591-f3156b071c50 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:55.882876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ccc3c-1b84-478e-83ee-3ffdecc77d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:55.883502 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ccc3c-1b84-478e-83ee-3ffdecc77d66 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:55.883785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ccc3c-1b84-478e-83ee-3ffdecc77d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:55.884080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ccc3c-1b84-478e-83ee-3ffdecc77d66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:55.885224 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ccc3c-1b84-478e-83ee-3ffdecc77d66 None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=8ef93155a40f4c69b320ca962d4e8868) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:55.885541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ccc3c-1b84-478e-83ee-3ffdecc77d66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:55.887105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ccc3c-1b84-478e-83ee-3ffdecc77d66 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:55.889203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:55.895616 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 279/538] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:55 2022] GET /identity/v3/role_assignments?user.id=8ef93155a40f4c69b320ca962d4e8868&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:55.903594 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ccc3c-1b84-478e-83ee-3ffdecc77d66 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:55.919763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 260/539] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:56.023720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ccc3c-1b84-478e-83ee-3ffdecc77d66 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:56.066761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd1f54-b6b9-44ec-890e-251b5d1c6bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:56.067738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd1f54-b6b9-44ec-890e-251b5d1c6bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:56.068245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd1f54-b6b9-44ec-890e-251b5d1c6bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:56.068718 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd1f54-b6b9-44ec-890e-251b5d1c6bd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:56.069585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd1f54-b6b9-44ec-890e-251b5d1c6bd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:56.070104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd1f54-b6b9-44ec-890e-251b5d1c6bd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:56.071762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd1f54-b6b9-44ec-890e-251b5d1c6bd7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:56.086307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 280/540] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:56.091630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94413a-53fa-4591-8104-c035d9cc19d6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:56.118968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:56.232502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f94413a-53fa-4591-8104-c035d9cc19d6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:56.267082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd1f54-b6b9-44ec-890e-251b5d1c6bd7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:56.289394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f51ae-386e-4145-a252-f84561ed51a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:56.290038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f51ae-386e-4145-a252-f84561ed51a3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:56.290323 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f51ae-386e-4145-a252-f84561ed51a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:56.290590 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f51ae-386e-4145-a252-f84561ed51a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:56.306372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f51ae-386e-4145-a252-f84561ed51a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:56.306372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f51ae-386e-4145-a252-f84561ed51a3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-cd6f51ae-386e-4145-a252-f84561ed51a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:16:56.320865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:56.321770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:56.322619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:16:56.322619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:16:56.322619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:16:56.322619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:16:56.322619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:16:56.322619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:16:56.322619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:16:56.322619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:56.322619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 261/541] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:56.332554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f51ae-386e-4145-a252-f84561ed51a3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:56.363697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:56.437039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd1f54-b6b9-44ec-890e-251b5d1c6bd7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:56.468435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:56.497732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331f091-469b-4136-8f8c-a88a04017934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:56.498280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331f091-469b-4136-8f8c-a88a04017934 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:56.498554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331f091-469b-4136-8f8c-a88a04017934 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:56.498826 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331f091-469b-4136-8f8c-a88a04017934 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:56.511536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f51ae-386e-4145-a252-f84561ed51a3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:56.520870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331f091-469b-4136-8f8c-a88a04017934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-b331f091-469b-4136-8f8c-a88a04017934 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:56.525656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:56.526532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:56.527327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:56.527327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:56.527327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:56.527327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:56.527327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:56.527327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:56.527327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:56.527327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:56.527327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:56.527327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:56.527327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:56.527327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:56.529136 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 281/542] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:16:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:16:56.535898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331f091-469b-4136-8f8c-a88a04017934 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:56.571124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b38721-843f-45e2-a2ac-155c5d57c7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:56.572601 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b38721-843f-45e2-a2ac-155c5d57c7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:56.573256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b38721-843f-45e2-a2ac-155c5d57c7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:56.573862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b38721-843f-45e2-a2ac-155c5d57c7e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:56.575011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b38721-843f-45e2-a2ac-155c5d57c7e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:56.575816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b38721-843f-45e2-a2ac-155c5d57c7e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:56.577927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b38721-843f-45e2-a2ac-155c5d57c7e0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:56.602785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 262/543] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:16:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:56.607387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b38721-843f-45e2-a2ac-155c5d57c7e0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:56.701288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331f091-469b-4136-8f8c-a88a04017934 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:56.748448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:56.749691 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b38721-843f-45e2-a2ac-155c5d57c7e0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:56.791802 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a4375-c81f-4e25-83d4-1270e67abe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:56.793154 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a4375-c81f-4e25-83d4-1270e67abe14 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:56.793578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a4375-c81f-4e25-83d4-1270e67abe14 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:56.793980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a4375-c81f-4e25-83d4-1270e67abe14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:56.794986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a4375-c81f-4e25-83d4-1270e67abe14 None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=0bf2a5de0e744505a1317ce9fa224643) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:56.795427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a4375-c81f-4e25-83d4-1270e67abe14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:56.797151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a4375-c81f-4e25-83d4-1270e67abe14 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:56.801221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:56.805508 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 263/544] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:56 2022] GET /identity/v3/role_assignments?user.id=0bf2a5de0e744505a1317ce9fa224643&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:56.816839 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539ec97-0131-42e5-bbbc-835a39ce9b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:56.817592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539ec97-0131-42e5-bbbc-835a39ce9b93 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:56.817876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539ec97-0131-42e5-bbbc-835a39ce9b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:56.818194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539ec97-0131-42e5-bbbc-835a39ce9b93 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:56.818882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539ec97-0131-42e5-bbbc-835a39ce9b93 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:56.819182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539ec97-0131-42e5-bbbc-835a39ce9b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:56.820712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539ec97-0131-42e5-bbbc-835a39ce9b93 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:56.840881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 282/545] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:16:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:56.846389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a4375-c81f-4e25-83d4-1270e67abe14 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:57.023360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a4375-c81f-4e25-83d4-1270e67abe14 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:57.083467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f3485-d5d3-4e00-92f9-7f26610a40fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:57.084163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3485-d5d3-4e00-92f9-7f26610a40fe None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:57.084452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3485-d5d3-4e00-92f9-7f26610a40fe None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:57.084728 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3485-d5d3-4e00-92f9-7f26610a40fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:57.115860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f3485-d5d3-4e00-92f9-7f26610a40fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:57.117366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f3485-d5d3-4e00-92f9-7f26610a40fe None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-476f3485-d5d3-4e00-92f9-7f26610a40fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:57.136624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:57.137581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:57.138430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:57.138430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:57.138430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:57.138430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 264/546] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:57.143442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539ec97-0131-42e5-bbbc-835a39ce9b93 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:57.301922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539ec97-0131-42e5-bbbc-835a39ce9b93 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:57.334286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741faa1c-9da3-4410-ae75-fcedc0850d9e None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:57.334803 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741faa1c-9da3-4410-ae75-fcedc0850d9e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:57.334803 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741faa1c-9da3-4410-ae75-fcedc0850d9e None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:57.337659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 265/547] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:16:57.339990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ccb2c-900b-4b03-ade8-e8683e9e9687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:57.340931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ccb2c-900b-4b03-ade8-e8683e9e9687 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:57.341417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ccb2c-900b-4b03-ade8-e8683e9e9687 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:57.341890 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ccb2c-900b-4b03-ade8-e8683e9e9687 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:57.342244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b228af-0c22-4947-941c-feb38d4a5344 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:57.342523 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b228af-0c22-4947-941c-feb38d4a5344 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:57.342784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b228af-0c22-4947-941c-feb38d4a5344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:57.343445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ccb2c-900b-4b03-ade8-e8683e9e9687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:57.343945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ccb2c-900b-4b03-ade8-e8683e9e9687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:57.345583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ccb2c-900b-4b03-ade8-e8683e9e9687 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:57.358967 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 283/548] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:57.365143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ccb2c-900b-4b03-ade8-e8683e9e9687 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:57.377080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-39b228af-0c22-4947-941c-feb38d4a5344 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:57.379504 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-39b228af-0c22-4947-941c-feb38d4a5344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:57.436652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:57.469526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:57.497049 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 266/549] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:57.498356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24444a9-ac59-4a61-af71-585e6b82907c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:57.498739 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24444a9-ac59-4a61-af71-585e6b82907c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:57.498902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24444a9-ac59-4a61-af71-585e6b82907c None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:57.504865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 267/550] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:57.506579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b181e48-1855-42c8-bcd5-e5d1e0c6b8f1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:57.506711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b181e48-1855-42c8-bcd5-e5d1e0c6b8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:57.506938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b181e48-1855-42c8-bcd5-e5d1e0c6b8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:57.509821 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ccb2c-900b-4b03-ade8-e8683e9e9687 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:57.525638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0b181e48-1855-42c8-bcd5-e5d1e0c6b8f1 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:57.526981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0b181e48-1855-42c8-bcd5-e5d1e0c6b8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:57.557870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a46d4-11e1-4718-953d-093f76502fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:57.558429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a46d4-11e1-4718-953d-093f76502fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:57.558705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a46d4-11e1-4718-953d-093f76502fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:57.559184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a46d4-11e1-4718-953d-093f76502fa5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:57.569651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a46d4-11e1-4718-953d-093f76502fa5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:57.571152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a46d4-11e1-4718-953d-093f76502fa5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5d7a46d4-11e1-4718-953d-093f76502fa5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:16:57.601220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:57.602405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:57.603349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:57.603349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:57.603349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:16:57.603349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:16:57.603349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:16:57.603349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:16:57.603349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:16:57.603349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:16:57.603349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 05:16:57.603349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:57.603349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 284/551] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:16:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:16:57.607843 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a46d4-11e1-4718-953d-093f76502fa5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:57.656342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 268/552] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:16:57.753743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:57.764300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a46d4-11e1-4718-953d-093f76502fa5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:57.816698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262ce5e-8e47-49e8-b7d2-6c311c0eed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:57.817308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262ce5e-8e47-49e8-b7d2-6c311c0eed27 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:57.817582 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262ce5e-8e47-49e8-b7d2-6c311c0eed27 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:57.817850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262ce5e-8e47-49e8-b7d2-6c311c0eed27 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:57.818632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262ce5e-8e47-49e8-b7d2-6c311c0eed27 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:57.818931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262ce5e-8e47-49e8-b7d2-6c311c0eed27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:57.820448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262ce5e-8e47-49e8-b7d2-6c311c0eed27 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:57.841940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 285/553] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 11 05:16:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:16:57.847875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b181e48-1855-42c8-bcd5-e5d1e0c6b8f1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:57.895330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262ce5e-8e47-49e8-b7d2-6c311c0eed27 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:58.012383 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b181e48-1855-42c8-bcd5-e5d1e0c6b8f1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:58.037851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:58.067158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262ce5e-8e47-49e8-b7d2-6c311c0eed27 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:58.068731 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2dd54-09a4-4b52-99b6-e484177c37aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:58.069508 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2dd54-09a4-4b52-99b6-e484177c37aa None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:58.070592 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2dd54-09a4-4b52-99b6-e484177c37aa None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:58.070861 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2dd54-09a4-4b52-99b6-e484177c37aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:58.110647 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8ee18-d9cd-4b47-a266-12cdcf07e603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:58.111286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8ee18-d9cd-4b47-a266-12cdcf07e603 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:58.111637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8ee18-d9cd-4b47-a266-12cdcf07e603 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:58.112438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8ee18-d9cd-4b47-a266-12cdcf07e603 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:58.125807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8ee18-d9cd-4b47-a266-12cdcf07e603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e3c8ee18-d9cd-4b47-a266-12cdcf07e603 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:58.132668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:58.133587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:58.134399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:58.134399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:58.134399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:58.134399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:58.134399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:58.134399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:58.134399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:58.134399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:58.134399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:58.134399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:58.134399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:58.134399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:58.134399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 286/554] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:16:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:58.135087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8ee18-d9cd-4b47-a266-12cdcf07e603 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:58.160330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:58.160638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2dd54-09a4-4b52-99b6-e484177c37aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:58.162247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2dd54-09a4-4b52-99b6-e484177c37aa None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:58.258860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8ee18-d9cd-4b47-a266-12cdcf07e603 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:58.294618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574092cf-1b04-48ab-89f3-42e2bfde7306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:58.295363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574092cf-1b04-48ab-89f3-42e2bfde7306 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:58.295647 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574092cf-1b04-48ab-89f3-42e2bfde7306 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:58.295912 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574092cf-1b04-48ab-89f3-42e2bfde7306 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:58.308091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 269/555] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:16:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:16:58.308971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2dd54-09a4-4b52-99b6-e484177c37aa None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:58.313521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574092cf-1b04-48ab-89f3-42e2bfde7306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-574092cf-1b04-48ab-89f3-42e2bfde7306 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:58.319469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:58.320748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:58.322240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:58.322240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:58.322240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:58.322240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:58.322240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:58.322240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:58.322240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:58.322240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:58.322240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:58.322240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:58.322240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:58.322240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:58.326516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 287/556] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:16:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:58.326516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574092cf-1b04-48ab-89f3-42e2bfde7306 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:58.328801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:58.415770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2dd54-09a4-4b52-99b6-e484177c37aa None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:58.431524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:58.457648 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574092cf-1b04-48ab-89f3-42e2bfde7306 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:58.461316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d8cb7-2591-4d2e-81e2-5a771451c360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:58.462218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d8cb7-2591-4d2e-81e2-5a771451c360 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:58.462665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d8cb7-2591-4d2e-81e2-5a771451c360 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:58.463090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d8cb7-2591-4d2e-81e2-5a771451c360 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:58.463922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d8cb7-2591-4d2e-81e2-5a771451c360 None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:58.464396 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d8cb7-2591-4d2e-81e2-5a771451c360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:58.466731 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d8cb7-2591-4d2e-81e2-5a771451c360 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:58.480361 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 270/557] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:16:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:58.481616 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d8cb7-2591-4d2e-81e2-5a771451c360 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:58.506490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed1db4-bc4f-44b2-9900-34263b3e9e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:58.507167 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed1db4-bc4f-44b2-9900-34263b3e9e14 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:58.507550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed1db4-bc4f-44b2-9900-34263b3e9e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:58.508145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed1db4-bc4f-44b2-9900-34263b3e9e14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:58.519429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed1db4-bc4f-44b2-9900-34263b3e9e14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:58.520934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed1db4-bc4f-44b2-9900-34263b3e9e14 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-f7ed1db4-bc4f-44b2-9900-34263b3e9e14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:58.536714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:58.537914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:58.539002 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:58.539002 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:58.539002 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:58.544011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 288/558] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:58.544011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed1db4-bc4f-44b2-9900-34263b3e9e14 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:58.597694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:58.621596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d8cb7-2591-4d2e-81e2-5a771451c360 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:58.652010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed1db4-bc4f-44b2-9900-34263b3e9e14 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:58.657267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b507efb-d935-4bb6-9960-53cf4fd6132a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:58.657884 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b507efb-d935-4bb6-9960-53cf4fd6132a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:58.658166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b507efb-d935-4bb6-9960-53cf4fd6132a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:58.658486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b507efb-d935-4bb6-9960-53cf4fd6132a None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:58.659146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b507efb-d935-4bb6-9960-53cf4fd6132a None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:58.659865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b507efb-d935-4bb6-9960-53cf4fd6132a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:58.661727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b507efb-d935-4bb6-9960-53cf4fd6132a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:58.675143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 271/559] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:16:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:58.675143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b507efb-d935-4bb6-9960-53cf4fd6132a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:58.690768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:58.698442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fe7b1-7ba4-4e23-8d5d-80bf13961014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:58.699225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fe7b1-7ba4-4e23-8d5d-80bf13961014 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:58.699506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fe7b1-7ba4-4e23-8d5d-80bf13961014 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:58.699776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fe7b1-7ba4-4e23-8d5d-80bf13961014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:58.712621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fe7b1-7ba4-4e23-8d5d-80bf13961014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:58.714080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fe7b1-7ba4-4e23-8d5d-80bf13961014 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-558fe7b1-7ba4-4e23-8d5d-80bf13961014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:58.731260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:58.737692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:58.738636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:58.738636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:58.738636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:58.738636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 289/560] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:58.738636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266adc47-da74-4869-a14b-6f5314f9c4a8 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:58.738636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266adc47-da74-4869-a14b-6f5314f9c4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:58.738636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266adc47-da74-4869-a14b-6f5314f9c4a8 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:58.738636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 290/561] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:58.738636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266adc47-da74-4869-a14b-6f5314f9c4a8 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:58.790390 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b507efb-d935-4bb6-9960-53cf4fd6132a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:58.828526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f39b5a-2897-4e58-9f10-d80eff94491e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:58.829704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f39b5a-2897-4e58-9f10-d80eff94491e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:58.829991 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f39b5a-2897-4e58-9f10-d80eff94491e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:58.830320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f39b5a-2897-4e58-9f10-d80eff94491e None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:58.831348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f39b5a-2897-4e58-9f10-d80eff94491e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:58.831665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f39b5a-2897-4e58-9f10-d80eff94491e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:58.833503 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f39b5a-2897-4e58-9f10-d80eff94491e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:58.836393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:58.851868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 272/562] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:58.851868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d98f27-cc30-4e59-a760-88f060ae4740 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:58.851868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d98f27-cc30-4e59-a760-88f060ae4740 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:58.851868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d98f27-cc30-4e59-a760-88f060ae4740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:58.863625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266adc47-da74-4869-a14b-6f5314f9c4a8 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:58.867981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c6d98f27-cc30-4e59-a760-88f060ae4740 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:58.869208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c6d98f27-cc30-4e59-a760-88f060ae4740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:58.902229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c9757-51f4-4b3c-8e4c-8c1d3c99ba3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:58.902960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c9757-51f4-4b3c-8e4c-8c1d3c99ba3f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:58.903242 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c9757-51f4-4b3c-8e4c-8c1d3c99ba3f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:58.903512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c9757-51f4-4b3c-8e4c-8c1d3c99ba3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:58.915347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c9757-51f4-4b3c-8e4c-8c1d3c99ba3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:58.917171 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c9757-51f4-4b3c-8e4c-8c1d3c99ba3f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-fc9c9757-51f4-4b3c-8e4c-8c1d3c99ba3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:16:58.933108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:58.934082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:16:58.934977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:16:58.934977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:16:58.934977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:58.934977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 291/563] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:58.935201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c9757-51f4-4b3c-8e4c-8c1d3c99ba3f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:58.952197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 273/564] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:58.952197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d98f27-cc30-4e59-a760-88f060ae4740 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:58.990739 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:59.056080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c9757-51f4-4b3c-8e4c-8c1d3c99ba3f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:59.063589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d98f27-cc30-4e59-a760-88f060ae4740 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:59.091826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:59.093807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f4776-1d78-4f5c-9631-42aa80ff6658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:59.094555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f4776-1d78-4f5c-9631-42aa80ff6658 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:59.094956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f4776-1d78-4f5c-9631-42aa80ff6658 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:59.095234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f4776-1d78-4f5c-9631-42aa80ff6658 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:59.095914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f4776-1d78-4f5c-9631-42aa80ff6658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:59.096395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f4776-1d78-4f5c-9631-42aa80ff6658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:59.097910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f4776-1d78-4f5c-9631-42aa80ff6658 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:59.104122 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140c57f-9a4d-4bc2-89b3-d49b456ac3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:59.105001 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140c57f-9a4d-4bc2-89b3-d49b456ac3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:59.105505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140c57f-9a4d-4bc2-89b3-d49b456ac3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:59.105992 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140c57f-9a4d-4bc2-89b3-d49b456ac3d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:59.113998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140c57f-9a4d-4bc2-89b3-d49b456ac3d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:59.115723 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140c57f-9a4d-4bc2-89b3-d49b456ac3d6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:59.116342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 292/565] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:59.117410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f4776-1d78-4f5c-9631-42aa80ff6658 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-8140c57f-9a4d-4bc2-89b3-d49b456ac3d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:16:59.133205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:59.134166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:59.135078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:16:59.135078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:16:59.135078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:16:59.135078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:16:59.135078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:16:59.135078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:16:59.135078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:16:59.135078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:59.135078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 274/566] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:16:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:59.136047 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140c57f-9a4d-4bc2-89b3-d49b456ac3d6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:59.235681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:59.245805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f4776-1d78-4f5c-9631-42aa80ff6658 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:59.250953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140c57f-9a4d-4bc2-89b3-d49b456ac3d6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:59.284264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97565816-87b8-4088-8510-9e256c288fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:59.284875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97565816-87b8-4088-8510-9e256c288fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:59.285108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7939147-be2b-480a-abe1-1fc6001a7b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:59.285232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97565816-87b8-4088-8510-9e256c288fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:59.285417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97565816-87b8-4088-8510-9e256c288fe9 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:59.285732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7939147-be2b-480a-abe1-1fc6001a7b3c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:59.286012 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7939147-be2b-480a-abe1-1fc6001a7b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:59.286112 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97565816-87b8-4088-8510-9e256c288fe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:59.286285 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7939147-be2b-480a-abe1-1fc6001a7b3c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:59.286376 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97565816-87b8-4088-8510-9e256c288fe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:59.287689 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97565816-87b8-4088-8510-9e256c288fe9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:59.293164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7939147-be2b-480a-abe1-1fc6001a7b3c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:59.294576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7939147-be2b-480a-abe1-1fc6001a7b3c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:59.303979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 293/567] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:59.303979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97565816-87b8-4088-8510-9e256c288fe9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-e7939147-be2b-480a-abe1-1fc6001a7b3c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:16:59.306834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:59.307896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:59.308895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:59.308895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:59.308895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:16:59.308895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:16:59.308895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:16:59.308895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:16:59.308895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:16:59.308895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:16:59.308895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 05:16:59.308895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:59.308895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 275/568] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:16:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:59.309496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d11328-fe20-448d-a55d-d12b22d3b1a8 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:59.309496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d11328-fe20-448d-a55d-d12b22d3b1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:59.309618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d11328-fe20-448d-a55d-d12b22d3b1a8 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:59.310750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 276/569] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:16:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:16:59.311427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d11328-fe20-448d-a55d-d12b22d3b1a8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:59.361388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:59.402551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97565816-87b8-4088-8510-9e256c288fe9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:59.416906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d11328-fe20-448d-a55d-d12b22d3b1a8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:59.435552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08496c-3319-4fbc-87e2-ce8a024598c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:59.436211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08496c-3319-4fbc-87e2-ce8a024598c5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:59.436499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08496c-3319-4fbc-87e2-ce8a024598c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:59.436873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08496c-3319-4fbc-87e2-ce8a024598c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:59.437562 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08496c-3319-4fbc-87e2-ce8a024598c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:59.437866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08496c-3319-4fbc-87e2-ce8a024598c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:59.439318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08496c-3319-4fbc-87e2-ce8a024598c5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:59.450710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dbfc7-7a75-4275-9d57-764a8e665664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:59.451335 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dbfc7-7a75-4275-9d57-764a8e665664 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:59.451620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dbfc7-7a75-4275-9d57-764a8e665664 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:59.451728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 294/570] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:16:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:59.451895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dbfc7-7a75-4275-9d57-764a8e665664 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:59.452492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08496c-3319-4fbc-87e2-ce8a024598c5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:59.465477 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917dbfc7-7a75-4275-9d57-764a8e665664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:16:59.467361 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-917dbfc7-7a75-4275-9d57-764a8e665664 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:16:59.471557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:59.472713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:59.473573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:59.473573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:59.473573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:59.473573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:59.473573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:16:59.473573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:16:59.473573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:16:59.473573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:16:59.473573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:16:59.473573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:16:59.473573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:16:59.473573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:16:59.473573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 277/571] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:16:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:59.474283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dbfc7-7a75-4275-9d57-764a8e665664 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:59.554057 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08496c-3319-4fbc-87e2-ce8a024598c5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:59.574609 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dbfc7-7a75-4275-9d57-764a8e665664 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:59.581958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:59.588251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc97da1-73d3-41a7-9915-ae48011f66e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:59.588880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc97da1-73d3-41a7-9915-ae48011f66e4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:59.589161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc97da1-73d3-41a7-9915-ae48011f66e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:59.589433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc97da1-73d3-41a7-9915-ae48011f66e4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:59.596514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc97da1-73d3-41a7-9915-ae48011f66e4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:59.597951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc97da1-73d3-41a7-9915-ae48011f66e4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:59.608882 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f49cde-6650-43b0-abeb-a17adc174928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-3cc97da1-73d3-41a7-9915-ae48011f66e4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:16:59.609194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:16:59.610259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:16:59.611233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:16:59.611233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:16:59.611233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:16:59.611233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:16:59.611233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:16:59.611233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:16:59.611233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:16:59.611233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:16:59.611233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:16:59.611233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:16:59.611923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49cde-6650-43b0-abeb-a17adc174928 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:59.611923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49cde-6650-43b0-abeb-a17adc174928 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:59.611923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f49cde-6650-43b0-abeb-a17adc174928 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:59.612665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 295/572] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 05:16:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:16:59.612765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49cde-6650-43b0-abeb-a17adc174928 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:59.612765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49cde-6650-43b0-abeb-a17adc174928 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:59.612936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014bf9b-0841-48da-a644-fd3801b0a979 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:59.612936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014bf9b-0841-48da-a644-fd3801b0a979 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:59.612936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014bf9b-0841-48da-a644-fd3801b0a979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:59.613221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f49cde-6650-43b0-abeb-a17adc174928 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:59.631722 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 278/573] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 11 05:16:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:59.632669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f49cde-6650-43b0-abeb-a17adc174928 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:59.633904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e014bf9b-0841-48da-a644-fd3801b0a979 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:16:59.635113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e014bf9b-0841-48da-a644-fd3801b0a979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:16:59.701516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 296/574] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:16:59.702355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014bf9b-0841-48da-a644-fd3801b0a979 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:59.717258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:59.731569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f49cde-6650-43b0-abeb-a17adc174928 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:59.766370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304550-3007-4207-b1d4-05779ddaef51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:59.766969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304550-3007-4207-b1d4-05779ddaef51 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:59.767245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304550-3007-4207-b1d4-05779ddaef51 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:59.767513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304550-3007-4207-b1d4-05779ddaef51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:59.768858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7304550-3007-4207-b1d4-05779ddaef51 None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=0a726949e8cc4cfdb25aa78f2761cff5) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:59.769164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7304550-3007-4207-b1d4-05779ddaef51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:59.770600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7304550-3007-4207-b1d4-05779ddaef51 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:59.788024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 279/575] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:16:59 2022] GET /identity/v3/role_assignments?user.id=0a726949e8cc4cfdb25aa78f2761cff5&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:59.788024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304550-3007-4207-b1d4-05779ddaef51 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:59.835767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:16:59.840157 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014bf9b-0841-48da-a644-fd3801b0a979 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:59.888375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbb96a-723f-4b60-841d-784acd3d7cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:59.889591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbb96a-723f-4b60-841d-784acd3d7cce None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:59.890060 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbb96a-723f-4b60-841d-784acd3d7cce None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:59.890800 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbb96a-723f-4b60-841d-784acd3d7cce None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:59.892190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbb96a-723f-4b60-841d-784acd3d7cce None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:59.893205 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbb96a-723f-4b60-841d-784acd3d7cce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:59.895652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbb96a-723f-4b60-841d-784acd3d7cce None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:59.909927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 297/576] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:16:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:16:59.911165 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbb96a-723f-4b60-841d-784acd3d7cce None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:16:59.948092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7304550-3007-4207-b1d4-05779ddaef51 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:16:59.991593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8a1cd-efb4-4f98-9b7e-ca3a8eafdbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:16:59.992242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8a1cd-efb4-4f98-9b7e-ca3a8eafdbe0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:16:59.992526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8a1cd-efb4-4f98-9b7e-ca3a8eafdbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:16:59.992796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8a1cd-efb4-4f98-9b7e-ca3a8eafdbe0 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:16:59.993534 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8a1cd-efb4-4f98-9b7e-ca3a8eafdbe0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:16:59.993879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8a1cd-efb4-4f98-9b7e-ca3a8eafdbe0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:16:59.995321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8a1cd-efb4-4f98-9b7e-ca3a8eafdbe0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:16:59.998216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:00.014209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 280/577] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 05:16:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:00.017065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8a1cd-efb4-4f98-9b7e-ca3a8eafdbe0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:00.038739 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbb96a-723f-4b60-841d-784acd3d7cce None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:00.077418 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efd1b5-c631-48bc-912a-d02199aa4665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:00.078461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efd1b5-c631-48bc-912a-d02199aa4665 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:00.078743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efd1b5-c631-48bc-912a-d02199aa4665 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:00.079268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efd1b5-c631-48bc-912a-d02199aa4665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:00.094005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31efd1b5-c631-48bc-912a-d02199aa4665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:00.096534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31efd1b5-c631-48bc-912a-d02199aa4665 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:00.107195 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-31efd1b5-c631-48bc-912a-d02199aa4665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:00.113535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:00.114480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:00.115367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:00.115367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:00.115367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:00.115367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 298/578] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:16:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:00.115598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efd1b5-c631-48bc-912a-d02199aa4665 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:00.129412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8a1cd-efb4-4f98-9b7e-ca3a8eafdbe0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:00.168068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74aa738-2166-46c5-8d2b-120f4935652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:00.169237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aa738-2166-46c5-8d2b-120f4935652e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:00.169730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aa738-2166-46c5-8d2b-120f4935652e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:00.170188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aa738-2166-46c5-8d2b-120f4935652e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:00.184527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74aa738-2166-46c5-8d2b-120f4935652e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:00.186946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74aa738-2166-46c5-8d2b-120f4935652e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-d74aa738-2166-46c5-8d2b-120f4935652e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:00.206835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:00.207869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:00.208814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:00.208814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:00.208814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:00.208814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 281/579] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:00.208814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74aa738-2166-46c5-8d2b-120f4935652e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:00.224297 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efd1b5-c631-48bc-912a-d02199aa4665 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:00.243482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:00.263866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071dde6-5fb1-443f-a0a7-9dfb463f08a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:00.264645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071dde6-5fb1-443f-a0a7-9dfb463f08a4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:00.264927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071dde6-5fb1-443f-a0a7-9dfb463f08a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:00.265314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071dde6-5fb1-443f-a0a7-9dfb463f08a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:00.276130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071dde6-5fb1-443f-a0a7-9dfb463f08a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e071dde6-5fb1-443f-a0a7-9dfb463f08a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:00.281716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:00.282639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:00.284058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:00.284058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:00.284058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:00.284058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:00.284058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:00.284058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:00.284058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:00.284058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:00.284058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:00.284058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:00.284058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:00.284058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:00.285997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 299/580] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:17:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:00.286941 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071dde6-5fb1-443f-a0a7-9dfb463f08a4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:00.324024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74aa738-2166-46c5-8d2b-120f4935652e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:00.359812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c6972-b698-40e4-96c2-396f0b1a7f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:00.360490 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c6972-b698-40e4-96c2-396f0b1a7f15 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:00.360753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c6972-b698-40e4-96c2-396f0b1a7f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:00.361031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c6972-b698-40e4-96c2-396f0b1a7f15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:00.368319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:00.372355 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c6972-b698-40e4-96c2-396f0b1a7f15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:00.374314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c6972-b698-40e4-96c2-396f0b1a7f15 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-4a6c6972-b698-40e4-96c2-396f0b1a7f15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:00.390160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:00.391320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:00.392262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:00.392262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:00.392262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:00.392262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 282/581] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:00.392540 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c6972-b698-40e4-96c2-396f0b1a7f15 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:00.406133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071dde6-5fb1-443f-a0a7-9dfb463f08a4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:00.445569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb777783-62ab-4aa0-8aef-797bd471ee51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:00.446108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb777783-62ab-4aa0-8aef-797bd471ee51 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:00.446390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb777783-62ab-4aa0-8aef-797bd471ee51 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:00.446661 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb777783-62ab-4aa0-8aef-797bd471ee51 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:00.447302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb777783-62ab-4aa0-8aef-797bd471ee51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:00.447607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb777783-62ab-4aa0-8aef-797bd471ee51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:00.449006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb777783-62ab-4aa0-8aef-797bd471ee51 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:00.465266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 300/582] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:00.465266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb777783-62ab-4aa0-8aef-797bd471ee51 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:00.510354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c6972-b698-40e4-96c2-396f0b1a7f15 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:00.511696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:00.548244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f5dfc-42dc-49d2-8511-b5a8b77d25a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:00.548855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f5dfc-42dc-49d2-8511-b5a8b77d25a2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:00.549945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f5dfc-42dc-49d2-8511-b5a8b77d25a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:00.550232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f5dfc-42dc-49d2-8511-b5a8b77d25a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:00.550881 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f5dfc-42dc-49d2-8511-b5a8b77d25a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:00.551189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f5dfc-42dc-49d2-8511-b5a8b77d25a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:00.552617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f5dfc-42dc-49d2-8511-b5a8b77d25a2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:00.565108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 283/583] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:00.566623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f5dfc-42dc-49d2-8511-b5a8b77d25a2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:00.576256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb777783-62ab-4aa0-8aef-797bd471ee51 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:00.613299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b2033-60ab-4150-a31b-3a72c5094600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:00.614028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b2033-60ab-4150-a31b-3a72c5094600 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:00.614309 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b2033-60ab-4150-a31b-3a72c5094600 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:00.614580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b2033-60ab-4150-a31b-3a72c5094600 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:00.615245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b2033-60ab-4150-a31b-3a72c5094600 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:00.615880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b2033-60ab-4150-a31b-3a72c5094600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:00.617312 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b2033-60ab-4150-a31b-3a72c5094600 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:00.623669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:00.626946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 301/584] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:17:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:00.633339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b2033-60ab-4150-a31b-3a72c5094600 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:00.684936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f5dfc-42dc-49d2-8511-b5a8b77d25a2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:00.722272 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9f628-7608-4676-9cd3-73026249ad7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:00.723146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9f628-7608-4676-9cd3-73026249ad7f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:00.723432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9f628-7608-4676-9cd3-73026249ad7f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:00.723713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9f628-7608-4676-9cd3-73026249ad7f None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:00.724400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9f628-7608-4676-9cd3-73026249ad7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:00.724708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9f628-7608-4676-9cd3-73026249ad7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:00.726083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9f628-7608-4676-9cd3-73026249ad7f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:00.743287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 284/585] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:00.743287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9f628-7608-4676-9cd3-73026249ad7f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:00.751050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:00.754676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b2033-60ab-4150-a31b-3a72c5094600 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:00.790857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fd892-6ee5-4cb2-add4-ddc6a646bc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:00.791491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fd892-6ee5-4cb2-add4-ddc6a646bc18 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:00.791774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fd892-6ee5-4cb2-add4-ddc6a646bc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:00.792159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fd892-6ee5-4cb2-add4-ddc6a646bc18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:00.799194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760fd892-6ee5-4cb2-add4-ddc6a646bc18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:00.801079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760fd892-6ee5-4cb2-add4-ddc6a646bc18 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-760fd892-6ee5-4cb2-add4-ddc6a646bc18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:00.813410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:00.814375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:00.815750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:00.815750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:00.815750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:00.815750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:00.815750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:00.815750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:00.815750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:00.815750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:00.815750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 302/586] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:00.816313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fd892-6ee5-4cb2-add4-ddc6a646bc18 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:00.862705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9f628-7608-4676-9cd3-73026249ad7f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:00.873925 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:00.899420 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ace3c-275c-4d48-bb09-61fa6c1ca75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:00.900716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ace3c-275c-4d48-bb09-61fa6c1ca75b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:00.901002 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ace3c-275c-4d48-bb09-61fa6c1ca75b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:00.901272 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ace3c-275c-4d48-bb09-61fa6c1ca75b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:00.909939 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ace3c-275c-4d48-bb09-61fa6c1ca75b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:00.911390 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ace3c-275c-4d48-bb09-61fa6c1ca75b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-063ace3c-275c-4d48-bb09-61fa6c1ca75b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:00.927282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:00.928343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:00.929304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:00.929304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:00.929304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:00.929304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:00.929304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:00.929304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:00.929304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:00.929304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:00.929304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 05:17:00.929304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:00.929304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 285/587] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:17:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:00.930434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ace3c-275c-4d48-bb09-61fa6c1ca75b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:00.937338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fd892-6ee5-4cb2-add4-ddc6a646bc18 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:00.973496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f9363-7836-4473-ad9e-6b19deac54fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:00.974162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f9363-7836-4473-ad9e-6b19deac54fb None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:00.974505 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f9363-7836-4473-ad9e-6b19deac54fb None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:00.975076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f9363-7836-4473-ad9e-6b19deac54fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:00.986546 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f9363-7836-4473-ad9e-6b19deac54fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:00.988471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f9363-7836-4473-ad9e-6b19deac54fb None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-b93f9363-7836-4473-ad9e-6b19deac54fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:01.004679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:01.005672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:01.007369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:01.007369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:01.007369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:01.007369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 303/588] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:01.007369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f9363-7836-4473-ad9e-6b19deac54fb None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:01.024214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:01.045621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ace3c-275c-4d48-bb09-61fa6c1ca75b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:01.086822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9f083-15be-44e0-bdd5-61243c96c144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:01.087449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9f083-15be-44e0-bdd5-61243c96c144 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:01.087725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9f083-15be-44e0-bdd5-61243c96c144 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:01.087991 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9f083-15be-44e0-bdd5-61243c96c144 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:01.094866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9f083-15be-44e0-bdd5-61243c96c144 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:01.096672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9f083-15be-44e0-bdd5-61243c96c144 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-04a9f083-15be-44e0-bdd5-61243c96c144 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:01.107759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:01.108877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:01.109824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:01.109824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:01.109824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:01.109824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:01.109824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:01.109824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:01.109824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:01.109824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:01.109824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 286/589] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:01.110351 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9f083-15be-44e0-bdd5-61243c96c144 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:01.119926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f9363-7836-4473-ad9e-6b19deac54fb None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:01.142783 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:01.159232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7b044-4835-4415-8450-6dc5d6bfdeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:01.159831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7b044-4835-4415-8450-6dc5d6bfdeda None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:01.160146 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7b044-4835-4415-8450-6dc5d6bfdeda None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:01.160420 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7b044-4835-4415-8450-6dc5d6bfdeda None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:01.161110 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7b044-4835-4415-8450-6dc5d6bfdeda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:01.161514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7b044-4835-4415-8450-6dc5d6bfdeda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:01.162863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7b044-4835-4415-8450-6dc5d6bfdeda None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:01.175071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 304/590] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:01.175971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7b044-4835-4415-8450-6dc5d6bfdeda None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:01.227666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9f083-15be-44e0-bdd5-61243c96c144 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:01.255780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:01.267259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35449f24-6900-4290-8d83-9281138011e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:01.269400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35449f24-6900-4290-8d83-9281138011e4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:01.270189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35449f24-6900-4290-8d83-9281138011e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:01.270613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35449f24-6900-4290-8d83-9281138011e4 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:01.271757 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35449f24-6900-4290-8d83-9281138011e4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:01.272530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35449f24-6900-4290-8d83-9281138011e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:01.274140 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35449f24-6900-4290-8d83-9281138011e4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:01.287003 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7b044-4835-4415-8450-6dc5d6bfdeda None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:01.295901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 287/591] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 11 05:17:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:01.297328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35449f24-6900-4290-8d83-9281138011e4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:01.324802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae993773-5a6c-4adf-a112-2ef9d8fd9f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:01.325573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae993773-5a6c-4adf-a112-2ef9d8fd9f9b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:01.325852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae993773-5a6c-4adf-a112-2ef9d8fd9f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:01.326116 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae993773-5a6c-4adf-a112-2ef9d8fd9f9b None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:01.326750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae993773-5a6c-4adf-a112-2ef9d8fd9f9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:01.327048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae993773-5a6c-4adf-a112-2ef9d8fd9f9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:01.328464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae993773-5a6c-4adf-a112-2ef9d8fd9f9b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:01.344860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 305/592] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:01.344860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae993773-5a6c-4adf-a112-2ef9d8fd9f9b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:01.399175 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:01.413121 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35449f24-6900-4290-8d83-9281138011e4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:01.447437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae993773-5a6c-4adf-a112-2ef9d8fd9f9b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:01.449803 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba364d-0d7e-4922-9808-848db4f16f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:01.450413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba364d-0d7e-4922-9808-848db4f16f97 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:01.450689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba364d-0d7e-4922-9808-848db4f16f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:01.450957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba364d-0d7e-4922-9808-848db4f16f97 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:01.451646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba364d-0d7e-4922-9808-848db4f16f97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:01.451948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba364d-0d7e-4922-9808-848db4f16f97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:01.453405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba364d-0d7e-4922-9808-848db4f16f97 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:01.471068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 288/593] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:01.471068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba364d-0d7e-4922-9808-848db4f16f97 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:01.482237 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:01.488858 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19932858-4d8f-4614-82cc-36f72a85b756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:01.489516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19932858-4d8f-4614-82cc-36f72a85b756 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:01.489792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19932858-4d8f-4614-82cc-36f72a85b756 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:01.490061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19932858-4d8f-4614-82cc-36f72a85b756 None admin] PATH_INFO: `/v3/projects/092dd30ff8b541e2a45947504831998a/users/0330c79c391d463b91f99e030c6dcc07/roles/7a06d87f01d940b38065f296fa63d0f7` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:01.505684 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19932858-4d8f-4614-82cc-36f72a85b756 None admin] RBAC: Authorizing `identity:create_grant(project_id=092dd30ff8b541e2a45947504831998a, user_id=0330c79c391d463b91f99e030c6dcc07, role_id=7a06d87f01d940b38065f296fa63d0f7)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:01.507122 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19932858-4d8f-4614-82cc-36f72a85b756 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:01.520830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 306/594] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:17:01 2022] PUT /identity/v3/projects/092dd30ff8b541e2a45947504831998a/users/0330c79c391d463b91f99e030c6dcc07/roles/7a06d87f01d940b38065f296fa63d0f7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:17:01.532980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19932858-4d8f-4614-82cc-36f72a85b756 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:01.592237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba364d-0d7e-4922-9808-848db4f16f97 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:01.632981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436a40f-9287-42a7-9da5-1adb94f9a3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:01.633662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436a40f-9287-42a7-9da5-1adb94f9a3ba None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:01.633998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436a40f-9287-42a7-9da5-1adb94f9a3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:01.634224 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436a40f-9287-42a7-9da5-1adb94f9a3ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:01.647247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436a40f-9287-42a7-9da5-1adb94f9a3ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:01.648793 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436a40f-9287-42a7-9da5-1adb94f9a3ba None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:01.659784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-8436a40f-9287-42a7-9da5-1adb94f9a3ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:01.668892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:01.669922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:01.671451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:01.671451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:01.671451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:01.671451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 289/595] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:01.671451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436a40f-9287-42a7-9da5-1adb94f9a3ba None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:01.686638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19932858-4d8f-4614-82cc-36f72a85b756 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:01.742800 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d936a2-0a8e-48e9-934e-d596cb8680ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:01.743788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d936a2-0a8e-48e9-934e-d596cb8680ed None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:01.744313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d936a2-0a8e-48e9-934e-d596cb8680ed None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:01.744800 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d936a2-0a8e-48e9-934e-d596cb8680ed None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:01.765150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d936a2-0a8e-48e9-934e-d596cb8680ed None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:01.768571 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d936a2-0a8e-48e9-934e-d596cb8680ed None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-f0d936a2-0a8e-48e9-934e-d596cb8680ed None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:01.782264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:01.783514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:01.784731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:01.784731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:01.784731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:01.784731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:01.784731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:01.784731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:01.784731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:01.784731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:01.784731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 05:17:01.784731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:01.785336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 307/596] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:17:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:01.786526 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d936a2-0a8e-48e9-934e-d596cb8680ed None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:01.834218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436a40f-9287-42a7-9da5-1adb94f9a3ba None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:01.838649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:01.878336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d310a-52e9-4a10-be2f-649ae2a75ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:01.879631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d310a-52e9-4a10-be2f-649ae2a75ef0 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:01.880191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d310a-52e9-4a10-be2f-649ae2a75ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:01.880674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d310a-52e9-4a10-be2f-649ae2a75ef0 None admin] PATH_INFO: `/v3/projects/092dd30ff8b541e2a45947504831998a/users/3bd5a9a09f034c05815c631a44d1cdd9/roles/7a06d87f01d940b38065f296fa63d0f7` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:01.899352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d310a-52e9-4a10-be2f-649ae2a75ef0 None admin] RBAC: Authorizing `identity:create_grant(project_id=092dd30ff8b541e2a45947504831998a, user_id=3bd5a9a09f034c05815c631a44d1cdd9, role_id=7a06d87f01d940b38065f296fa63d0f7)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:01.901238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d310a-52e9-4a10-be2f-649ae2a75ef0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:01.914263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d936a2-0a8e-48e9-934e-d596cb8680ed None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:01.917293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 290/597] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:17:01 2022] PUT /identity/v3/projects/092dd30ff8b541e2a45947504831998a/users/3bd5a9a09f034c05815c631a44d1cdd9/roles/7a06d87f01d940b38065f296fa63d0f7 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:17:01.918349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d310a-52e9-4a10-be2f-649ae2a75ef0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:01.953278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa281af-de00-4149-a762-337063887c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:01.954066 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa281af-de00-4149-a762-337063887c70 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:01.954442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa281af-de00-4149-a762-337063887c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:01.954813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa281af-de00-4149-a762-337063887c70 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:01.955651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa281af-de00-4149-a762-337063887c70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:01.956101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa281af-de00-4149-a762-337063887c70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:01.957779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa281af-de00-4149-a762-337063887c70 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:01.972385 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 308/598] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:01.985149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa281af-de00-4149-a762-337063887c70 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:01.986063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:02.045136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d310a-52e9-4a10-be2f-649ae2a75ef0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:02.086805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188afd1-ad22-439f-899c-2db36822cca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:02.087832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188afd1-ad22-439f-899c-2db36822cca5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:02.088382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188afd1-ad22-439f-899c-2db36822cca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:02.089380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188afd1-ad22-439f-899c-2db36822cca5 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:02.090483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188afd1-ad22-439f-899c-2db36822cca5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:02.090964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188afd1-ad22-439f-899c-2db36822cca5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:02.092754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188afd1-ad22-439f-899c-2db36822cca5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:02.116039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa281af-de00-4149-a762-337063887c70 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:02.116557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 291/599] 10.222.0.33 () {56 vars in 1139 bytes} [Sun Sep 11 05:17:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:02.118026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:02.119368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70216d-33ab-4b19-ba01-4a620ce71f0b None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:02.119904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70216d-33ab-4b19-ba01-4a620ce71f0b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:02.120400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70216d-33ab-4b19-ba01-4a620ce71f0b None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:02.123325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 292/600] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:02.124312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70216d-33ab-4b19-ba01-4a620ce71f0b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:02.159467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764e511-4015-40bc-a0f8-48919455d4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:02.160306 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e511-4015-40bc-a0f8-48919455d4ff None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:02.160694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e511-4015-40bc-a0f8-48919455d4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:02.161058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e511-4015-40bc-a0f8-48919455d4ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:02.170135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764e511-4015-40bc-a0f8-48919455d4ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:02.171820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764e511-4015-40bc-a0f8-48919455d4ff None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-b764e511-4015-40bc-a0f8-48919455d4ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:02.185479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:02.186593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:02.187716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:02.187716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:02.187716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:02.187716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:02.187716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:02.187716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:02.187716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:02.187716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:02.189889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 309/601] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:02.192081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d926e27-05c1-4b96-9f95-92016ef566b0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:02.192545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d926e27-05c1-4b96-9f95-92016ef566b0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:02.192979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d926e27-05c1-4b96-9f95-92016ef566b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:02.212258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2d926e27-05c1-4b96-9f95-92016ef566b0 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:02.213696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2d926e27-05c1-4b96-9f95-92016ef566b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:02.247243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70216d-33ab-4b19-ba01-4a620ce71f0b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:02.257677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:02.295611 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 310/602] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:17:02.296158 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7651416-1a24-4254-be6c-0ae4d90985f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:02.296313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d926e27-05c1-4b96-9f95-92016ef566b0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:02.297088 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7651416-1a24-4254-be6c-0ae4d90985f3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:02.297522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7651416-1a24-4254-be6c-0ae4d90985f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:02.297945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7651416-1a24-4254-be6c-0ae4d90985f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:02.320637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7651416-1a24-4254-be6c-0ae4d90985f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:02.322374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7651416-1a24-4254-be6c-0ae4d90985f3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-e7651416-1a24-4254-be6c-0ae4d90985f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:02.359846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:02.361092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:02.362117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:02.362117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:02.362117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:02.363501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 293/603] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:02.369039 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7651416-1a24-4254-be6c-0ae4d90985f3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:02.482554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:02.500658 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d926e27-05c1-4b96-9f95-92016ef566b0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:02.573201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d916c-4296-4cd1-86a6-de8523dd406a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:02.573816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d916c-4296-4cd1-86a6-de8523dd406a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:02.574098 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d916c-4296-4cd1-86a6-de8523dd406a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:02.574377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d916c-4296-4cd1-86a6-de8523dd406a None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:02.575071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d916c-4296-4cd1-86a6-de8523dd406a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:02.575376 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d916c-4296-4cd1-86a6-de8523dd406a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:02.576879 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d916c-4296-4cd1-86a6-de8523dd406a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:02.578883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7651416-1a24-4254-be6c-0ae4d90985f3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:02.589590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 311/604] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:02.590480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d916c-4296-4cd1-86a6-de8523dd406a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:02.616508 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85151eeb-8509-4823-a085-4dffa43c8629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:02.617685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85151eeb-8509-4823-a085-4dffa43c8629 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:02.618155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85151eeb-8509-4823-a085-4dffa43c8629 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:02.618589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85151eeb-8509-4823-a085-4dffa43c8629 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:02.619378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85151eeb-8509-4823-a085-4dffa43c8629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:02.619842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85151eeb-8509-4823-a085-4dffa43c8629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:02.621404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85151eeb-8509-4823-a085-4dffa43c8629 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:02.635087 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 294/605] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:02.636165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85151eeb-8509-4823-a085-4dffa43c8629 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:02.658967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:02.695343 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d916c-4296-4cd1-86a6-de8523dd406a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:02.732163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63a2c5-7897-428c-a0bf-7220d1f96654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:02.732802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63a2c5-7897-428c-a0bf-7220d1f96654 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:02.733417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63a2c5-7897-428c-a0bf-7220d1f96654 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:02.733417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63a2c5-7897-428c-a0bf-7220d1f96654 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:02.744653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63a2c5-7897-428c-a0bf-7220d1f96654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:02.746684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85151eeb-8509-4823-a085-4dffa43c8629 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-6f63a2c5-7897-428c-a0bf-7220d1f96654 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:02.749963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:02.751094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:02.752755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:02.752755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:02.752755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:02.752755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:02.752755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:02.752755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:02.752755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:02.752755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:02.752755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:02.752755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:02.752755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:02.752755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:02.756251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 312/606] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:17:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:02.757125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63a2c5-7897-428c-a0bf-7220d1f96654 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:02.773223 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:02.790050 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e6e35-9fd7-49ee-93ac-2978441c9898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:02.790985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e6e35-9fd7-49ee-93ac-2978441c9898 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:02.793761 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e6e35-9fd7-49ee-93ac-2978441c9898 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:02.794216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e6e35-9fd7-49ee-93ac-2978441c9898 None admin] PATH_INFO: `/v3/projects/092dd30ff8b541e2a45947504831998a/users/8ef93155a40f4c69b320ca962d4e8868/roles/7a06d87f01d940b38065f296fa63d0f7` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:02.813445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e6e35-9fd7-49ee-93ac-2978441c9898 None admin] RBAC: Authorizing `identity:create_grant(project_id=092dd30ff8b541e2a45947504831998a, user_id=8ef93155a40f4c69b320ca962d4e8868, role_id=7a06d87f01d940b38065f296fa63d0f7)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:02.815629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e6e35-9fd7-49ee-93ac-2978441c9898 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:02.832280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 295/607] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:17:02 2022] PUT /identity/v3/projects/092dd30ff8b541e2a45947504831998a/users/8ef93155a40f4c69b320ca962d4e8868/roles/7a06d87f01d940b38065f296fa63d0f7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:17:02.833396 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e6e35-9fd7-49ee-93ac-2978441c9898 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:02.867450 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63a2c5-7897-428c-a0bf-7220d1f96654 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:02.906107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c30e0-03d2-4e23-a8d3-81fff9af6419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:02.906797 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c30e0-03d2-4e23-a8d3-81fff9af6419 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:02.907080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c30e0-03d2-4e23-a8d3-81fff9af6419 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:02.907397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c30e0-03d2-4e23-a8d3-81fff9af6419 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:02.915583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c30e0-03d2-4e23-a8d3-81fff9af6419 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:02.917111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c30e0-03d2-4e23-a8d3-81fff9af6419 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:02.919105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-023c30e0-03d2-4e23-a8d3-81fff9af6419 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:02.929712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:02.930851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:02.932052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:02.932052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:02.932052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:02.932052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:02.932052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:02.932052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:02.932052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:02.932052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:02.932052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 313/608] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:02.936797 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c30e0-03d2-4e23-a8d3-81fff9af6419 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:02.958106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e6e35-9fd7-49ee-93ac-2978441c9898 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:03.003132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e819bc9-5631-4f69-aa21-2e4da9ad6915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:03.004207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e819bc9-5631-4f69-aa21-2e4da9ad6915 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:03.004594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e819bc9-5631-4f69-aa21-2e4da9ad6915 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:03.004970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e819bc9-5631-4f69-aa21-2e4da9ad6915 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:03.005749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e819bc9-5631-4f69-aa21-2e4da9ad6915 None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:03.006158 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e819bc9-5631-4f69-aa21-2e4da9ad6915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:03.008590 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e819bc9-5631-4f69-aa21-2e4da9ad6915 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:03.021847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 296/609] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:17:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:03.027444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e819bc9-5631-4f69-aa21-2e4da9ad6915 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:03.029061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:03.067514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c30e0-03d2-4e23-a8d3-81fff9af6419 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:03.105457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb9403-db44-4759-9e28-f8d43d681e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:03.106524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb9403-db44-4759-9e28-f8d43d681e9a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:03.107030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb9403-db44-4759-9e28-f8d43d681e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:03.107490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb9403-db44-4759-9e28-f8d43d681e9a None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:03.108435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb9403-db44-4759-9e28-f8d43d681e9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:03.108951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb9403-db44-4759-9e28-f8d43d681e9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:03.110687 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb9403-db44-4759-9e28-f8d43d681e9a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:03.125744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 314/610] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:03.133226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb9403-db44-4759-9e28-f8d43d681e9a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:03.141046 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e819bc9-5631-4f69-aa21-2e4da9ad6915 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:03.179583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:03.185169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd04d32-0709-47e8-89c6-5b28e9c51717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:03.185643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd04d32-0709-47e8-89c6-5b28e9c51717 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:03.185937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd04d32-0709-47e8-89c6-5b28e9c51717 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:03.186231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd04d32-0709-47e8-89c6-5b28e9c51717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:03.198130 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd04d32-0709-47e8-89c6-5b28e9c51717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:03.199739 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd04d32-0709-47e8-89c6-5b28e9c51717 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-4dd04d32-0709-47e8-89c6-5b28e9c51717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:03.216022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:03.217031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:03.217953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:03.217953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:03.217953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:03.217953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 297/611] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:03.224300 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd04d32-0709-47e8-89c6-5b28e9c51717 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:03.284971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb9403-db44-4759-9e28-f8d43d681e9a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:03.326454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:03.338032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd04d32-0709-47e8-89c6-5b28e9c51717 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:03.340100 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bafa1-e8e1-418b-abb7-39ca335a7b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:03.340772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bafa1-e8e1-418b-abb7-39ca335a7b7e None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:03.341047 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bafa1-e8e1-418b-abb7-39ca335a7b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:03.341315 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bafa1-e8e1-418b-abb7-39ca335a7b7e None admin] PATH_INFO: `/v3/projects/092dd30ff8b541e2a45947504831998a/users/0bf2a5de0e744505a1317ce9fa224643/roles/7a06d87f01d940b38065f296fa63d0f7` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:03.357860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bafa1-e8e1-418b-abb7-39ca335a7b7e None admin] RBAC: Authorizing `identity:create_grant(project_id=092dd30ff8b541e2a45947504831998a, user_id=0bf2a5de0e744505a1317ce9fa224643, role_id=7a06d87f01d940b38065f296fa63d0f7)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:03.359232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bafa1-e8e1-418b-abb7-39ca335a7b7e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:03.382166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6193b2-8188-4b02-a057-a7a314af5588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:03.383043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6193b2-8188-4b02-a057-a7a314af5588 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:03.383492 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6193b2-8188-4b02-a057-a7a314af5588 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:03.383921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6193b2-8188-4b02-a057-a7a314af5588 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:03.384750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6193b2-8188-4b02-a057-a7a314af5588 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:03.385206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6193b2-8188-4b02-a057-a7a314af5588 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:03.386451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 315/612] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:17:03 2022] PUT /identity/v3/projects/092dd30ff8b541e2a45947504831998a/users/0bf2a5de0e744505a1317ce9fa224643/roles/7a06d87f01d940b38065f296fa63d0f7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:17:03.386929 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6193b2-8188-4b02-a057-a7a314af5588 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:03.400815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 298/613] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:03.406650 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bafa1-e8e1-418b-abb7-39ca335a7b7e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:03.523099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bafa1-e8e1-418b-abb7-39ca335a7b7e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:03.560955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a7b5a-fa24-488d-bdd9-cf130790c8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:03.561611 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a7b5a-fa24-488d-bdd9-cf130790c8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:03.561890 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a7b5a-fa24-488d-bdd9-cf130790c8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:03.562161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a7b5a-fa24-488d-bdd9-cf130790c8d3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:03.570795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a7b5a-fa24-488d-bdd9-cf130790c8d3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:03.572469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a7b5a-fa24-488d-bdd9-cf130790c8d3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-a71a7b5a-fa24-488d-bdd9-cf130790c8d3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:03.586227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:03.587480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:03.588692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:03.588692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:03.588692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:03.588692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:03.588692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:03.588692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:03.588692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:03.588692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:03.588692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 05:17:03.588692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:03.588692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 316/614] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:17:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:03.595380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6193b2-8188-4b02-a057-a7a314af5588 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:03.665256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:03.727319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6193b2-8188-4b02-a057-a7a314af5588 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:03.776264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590fc00-323d-46ed-bf26-7238fb8e3ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:03.777118 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590fc00-323d-46ed-bf26-7238fb8e3ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:03.777583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590fc00-323d-46ed-bf26-7238fb8e3ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:03.778007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590fc00-323d-46ed-bf26-7238fb8e3ee0 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:03.778927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590fc00-323d-46ed-bf26-7238fb8e3ee0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:03.779381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590fc00-323d-46ed-bf26-7238fb8e3ee0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:03.780994 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590fc00-323d-46ed-bf26-7238fb8e3ee0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:03.807605 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 299/615] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 11 05:17:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:03.813302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a7b5a-fa24-488d-bdd9-cf130790c8d3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:03.873099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:03.933826 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a7b5a-fa24-488d-bdd9-cf130790c8d3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:03.969368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa3c51-afbf-497d-8890-2124417199aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:03.970034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa3c51-afbf-497d-8890-2124417199aa None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:03.970410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa3c51-afbf-497d-8890-2124417199aa None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:03.971278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa3c51-afbf-497d-8890-2124417199aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:03.984032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa3c51-afbf-497d-8890-2124417199aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:03.985525 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa3c51-afbf-497d-8890-2124417199aa None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-38fa3c51-afbf-497d-8890-2124417199aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:04.001813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:04.003390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:04.005513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:04.005513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:04.005513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:04.005513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 317/616] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:04.013109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590fc00-323d-46ed-bf26-7238fb8e3ee0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:04.147401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5f263-35d6-4b82-ac9c-9e0a0cf5f2e0 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:04.147684 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5f263-35d6-4b82-ac9c-9e0a0cf5f2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:04.147958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5f263-35d6-4b82-ac9c-9e0a0cf5f2e0 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:04.149447 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 318/617] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:04.154452 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71d1fa-e4b2-400d-80ae-3303a7e4daa6 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:04.154932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71d1fa-e4b2-400d-80ae-3303a7e4daa6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:04.155370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71d1fa-e4b2-400d-80ae-3303a7e4daa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:04.176127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0e71d1fa-e4b2-400d-80ae-3303a7e4daa6 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:04.177574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0e71d1fa-e4b2-400d-80ae-3303a7e4daa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:04.179597 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590fc00-323d-46ed-bf26-7238fb8e3ee0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:04.188104 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:04.254299 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835447c-83f3-4f65-98d8-44780ca2aea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:04.254975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835447c-83f3-4f65-98d8-44780ca2aea7 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:04.255260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835447c-83f3-4f65-98d8-44780ca2aea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:04.255533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835447c-83f3-4f65-98d8-44780ca2aea7 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:04.256253 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835447c-83f3-4f65-98d8-44780ca2aea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:04.256568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835447c-83f3-4f65-98d8-44780ca2aea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:04.258010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835447c-83f3-4f65-98d8-44780ca2aea7 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:04.286286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 300/618] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:04.291674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835447c-83f3-4f65-98d8-44780ca2aea7 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:04.295662 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 319/619] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:17:04.435087 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835447c-83f3-4f65-98d8-44780ca2aea7 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:04.487820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f3474-44e2-4219-b7d0-5f953e0d5520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:04.488403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f3474-44e2-4219-b7d0-5f953e0d5520 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:04.488742 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f3474-44e2-4219-b7d0-5f953e0d5520 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:04.489010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f3474-44e2-4219-b7d0-5f953e0d5520 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:04.501020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f3474-44e2-4219-b7d0-5f953e0d5520 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:04.502661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f3474-44e2-4219-b7d0-5f953e0d5520 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-609f3474-44e2-4219-b7d0-5f953e0d5520 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:04.517417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:04.518974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:04.519867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:04.519867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:04.519867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:04.519867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:04.519867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:04.519867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:04.519867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:04.519867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:04.519867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 301/620] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:04.526529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71d1fa-e4b2-400d-80ae-3303a7e4daa6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:04.567742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:04.574181 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4692826-e7d4-4119-b72e-1f32c5171d49 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:04.574467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4692826-e7d4-4119-b72e-1f32c5171d49 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:04.574744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4692826-e7d4-4119-b72e-1f32c5171d49 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:04.576085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 302/621] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:04.582071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afacca-c8f5-4015-91f5-f111084d517e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:04.582349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afacca-c8f5-4015-91f5-f111084d517e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:04.582621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afacca-c8f5-4015-91f5-f111084d517e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:04.611036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d1afacca-c8f5-4015-91f5-f111084d517e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:04.612805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d1afacca-c8f5-4015-91f5-f111084d517e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:04.654819 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:04.679340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71d1fa-e4b2-400d-80ae-3303a7e4daa6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:04.724519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39925959-d460-4567-a5be-5d95749cf61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:04.725153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39925959-d460-4567-a5be-5d95749cf61f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:04.725433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39925959-d460-4567-a5be-5d95749cf61f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:04.725706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39925959-d460-4567-a5be-5d95749cf61f None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:04.726426 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39925959-d460-4567-a5be-5d95749cf61f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:04.726735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39925959-d460-4567-a5be-5d95749cf61f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:04.731061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39925959-d460-4567-a5be-5d95749cf61f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:04.753969 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 320/622] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:04.754752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39925959-d460-4567-a5be-5d95749cf61f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:04.768765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 303/623] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:17:04.770178 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afacca-c8f5-4015-91f5-f111084d517e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:04.912362 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afacca-c8f5-4015-91f5-f111084d517e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:04.942135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:04.961864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39925959-d460-4567-a5be-5d95749cf61f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:04.999256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389f054-5aad-43e5-8c5e-d1ecae3fde1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:05.000284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389f054-5aad-43e5-8c5e-d1ecae3fde1e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:05.001223 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389f054-5aad-43e5-8c5e-d1ecae3fde1e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:05.001735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389f054-5aad-43e5-8c5e-d1ecae3fde1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:05.014139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389f054-5aad-43e5-8c5e-d1ecae3fde1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-2389f054-5aad-43e5-8c5e-d1ecae3fde1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:05.020825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:05.021752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:05.022599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:05.022599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:05.022599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:05.022599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:05.022599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:05.022599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:05.022599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:05.022599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:05.022599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:05.022599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:05.022599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:05.022599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:05.026374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 321/624] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:17:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:05.031663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389f054-5aad-43e5-8c5e-d1ecae3fde1e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:05.033275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0aec0-e502-4dc2-85e7-cee399ecb45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:05.034046 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0aec0-e502-4dc2-85e7-cee399ecb45d None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:05.034325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0aec0-e502-4dc2-85e7-cee399ecb45d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:05.034595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0aec0-e502-4dc2-85e7-cee399ecb45d None admin] PATH_INFO: `/v3/projects/092dd30ff8b541e2a45947504831998a/users/0a726949e8cc4cfdb25aa78f2761cff5/roles/7a06d87f01d940b38065f296fa63d0f7` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:05.084887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0aec0-e502-4dc2-85e7-cee399ecb45d None admin] RBAC: Authorizing `identity:create_grant(project_id=092dd30ff8b541e2a45947504831998a, user_id=0a726949e8cc4cfdb25aa78f2761cff5, role_id=7a06d87f01d940b38065f296fa63d0f7)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:05.086343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0aec0-e502-4dc2-85e7-cee399ecb45d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:05.118380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 304/625] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:17:04 2022] PUT /identity/v3/projects/092dd30ff8b541e2a45947504831998a/users/0a726949e8cc4cfdb25aa78f2761cff5/roles/7a06d87f01d940b38065f296fa63d0f7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:17:05.133102 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0aec0-e502-4dc2-85e7-cee399ecb45d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:05.142295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:05.159987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389f054-5aad-43e5-8c5e-d1ecae3fde1e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:05.199793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b4a90-438f-408c-8af0-d3115ecfc235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:05.200949 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b4a90-438f-408c-8af0-d3115ecfc235 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:05.201514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b4a90-438f-408c-8af0-d3115ecfc235 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:05.202009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b4a90-438f-408c-8af0-d3115ecfc235 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:05.202929 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b4a90-438f-408c-8af0-d3115ecfc235 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:05.203495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b4a90-438f-408c-8af0-d3115ecfc235 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:05.205928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b4a90-438f-408c-8af0-d3115ecfc235 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:05.216212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 322/626] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:17:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:05.222275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b4a90-438f-408c-8af0-d3115ecfc235 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:05.243414 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:05.376837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0aec0-e502-4dc2-85e7-cee399ecb45d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:05.413465 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b4a90-438f-408c-8af0-d3115ecfc235 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:05.424358 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7f1fa-6b84-4b01-9ed5-6e2419660820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:05.426824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7f1fa-6b84-4b01-9ed5-6e2419660820 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:05.426824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7f1fa-6b84-4b01-9ed5-6e2419660820 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:05.426824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7f1fa-6b84-4b01-9ed5-6e2419660820 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:05.452248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7f1fa-6b84-4b01-9ed5-6e2419660820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-36f7f1fa-6b84-4b01-9ed5-6e2419660820 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:05.458495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:05.459594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:05.460619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:05.460619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:05.460619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:05.460619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:05.460619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:05.460619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:05.460619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:05.460619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:05.460619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:05.460619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:05.460619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:05.460619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:05.464741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 305/627] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:17:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:05.473026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7f1fa-6b84-4b01-9ed5-6e2419660820 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:05.490336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7910fa-ee00-47a8-9ae5-86c6e73d4f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:05.490979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7910fa-ee00-47a8-9ae5-86c6e73d4f68 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:05.491257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7910fa-ee00-47a8-9ae5-86c6e73d4f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:05.491534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7910fa-ee00-47a8-9ae5-86c6e73d4f68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:05.518294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7910fa-ee00-47a8-9ae5-86c6e73d4f68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:05.521904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7910fa-ee00-47a8-9ae5-86c6e73d4f68 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-8c7910fa-ee00-47a8-9ae5-86c6e73d4f68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:05.557602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:05.559059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:05.561211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:05.561211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:05.561211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:05.567350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:05.568356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 323/628] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:05.569343 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7910fa-ee00-47a8-9ae5-86c6e73d4f68 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:05.646049 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7f1fa-6b84-4b01-9ed5-6e2419660820 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:05.698192 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25a0e3-e503-48ea-b09c-e318b5e5fb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:05.699188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25a0e3-e503-48ea-b09c-e318b5e5fb43 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:05.699529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25a0e3-e503-48ea-b09c-e318b5e5fb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:05.699806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25a0e3-e503-48ea-b09c-e318b5e5fb43 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:05.700549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25a0e3-e503-48ea-b09c-e318b5e5fb43 None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:05.700857 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25a0e3-e503-48ea-b09c-e318b5e5fb43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:05.702237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25a0e3-e503-48ea-b09c-e318b5e5fb43 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:05.710030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:05.720034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 306/629] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:17:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:05.720034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383580b-6948-4038-ae57-942820ddfabf None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:05.720034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383580b-6948-4038-ae57-942820ddfabf None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:05.720034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383580b-6948-4038-ae57-942820ddfabf None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:05.720034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 307/630] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:05.722088 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383580b-6948-4038-ae57-942820ddfabf None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:05.774411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7910fa-ee00-47a8-9ae5-86c6e73d4f68 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:05.818618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27045ec-29cd-4853-b60e-8d4b1ffbb291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:05.819455 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27045ec-29cd-4853-b60e-8d4b1ffbb291 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:05.819905 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27045ec-29cd-4853-b60e-8d4b1ffbb291 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:05.820338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27045ec-29cd-4853-b60e-8d4b1ffbb291 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:05.821124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27045ec-29cd-4853-b60e-8d4b1ffbb291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:05.821571 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27045ec-29cd-4853-b60e-8d4b1ffbb291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:05.823118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27045ec-29cd-4853-b60e-8d4b1ffbb291 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:05.837821 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 324/631] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:05.839391 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c563c-d4d1-414a-8576-959b7fbbb135 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:05.839828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c563c-d4d1-414a-8576-959b7fbbb135 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:05.840286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c563c-d4d1-414a-8576-959b7fbbb135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:05.853967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383580b-6948-4038-ae57-942820ddfabf None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:05.862380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d08c563c-d4d1-414a-8576-959b7fbbb135 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:05.863973 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d08c563c-d4d1-414a-8576-959b7fbbb135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:05.884404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:05.902453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86aafe-d67b-4f87-a1ea-7bf8ea6a7f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:05.903005 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86aafe-d67b-4f87-a1ea-7bf8ea6a7f5d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:05.903280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86aafe-d67b-4f87-a1ea-7bf8ea6a7f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:05.903551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86aafe-d67b-4f87-a1ea-7bf8ea6a7f5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:05.917518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86aafe-d67b-4f87-a1ea-7bf8ea6a7f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:05.919147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86aafe-d67b-4f87-a1ea-7bf8ea6a7f5d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-7c86aafe-d67b-4f87-a1ea-7bf8ea6a7f5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:05.939026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:05.940631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:05.941533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:05.941533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:05.941533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:05.945822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 308/632] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:05.952170 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86aafe-d67b-4f87-a1ea-7bf8ea6a7f5d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:05.952325 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 325/633] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:17:05.952325 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c563c-d4d1-414a-8576-959b7fbbb135 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:06.062735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:06.124027 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c563c-d4d1-414a-8576-959b7fbbb135 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:06.146175 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86aafe-d67b-4f87-a1ea-7bf8ea6a7f5d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:06.160526 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d50b70-8654-403b-a886-3c6bae93b009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:06.161056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d50b70-8654-403b-a886-3c6bae93b009 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:06.161403 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d50b70-8654-403b-a886-3c6bae93b009 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:06.161588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d50b70-8654-403b-a886-3c6bae93b009 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:06.162233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d50b70-8654-403b-a886-3c6bae93b009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:06.162529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d50b70-8654-403b-a886-3c6bae93b009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:06.164091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d50b70-8654-403b-a886-3c6bae93b009 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:06.180077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 326/634] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:06.181369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109206e0-6ef2-41b7-b22f-3ce0bdb57eca None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:06.181655 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109206e0-6ef2-41b7-b22f-3ce0bdb57eca None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:06.181930 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109206e0-6ef2-41b7-b22f-3ce0bdb57eca None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:06.184287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 327/635] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:06.185152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109206e0-6ef2-41b7-b22f-3ce0bdb57eca None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:06.200143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022a049-8047-464d-a7a7-3c490c627139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:06.200792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022a049-8047-464d-a7a7-3c490c627139 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:06.201493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022a049-8047-464d-a7a7-3c490c627139 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:06.201776 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022a049-8047-464d-a7a7-3c490c627139 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:06.212066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022a049-8047-464d-a7a7-3c490c627139 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:06.214085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022a049-8047-464d-a7a7-3c490c627139 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-6022a049-8047-464d-a7a7-3c490c627139 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:06.229887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:06.230915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:06.231883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:06.231883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:06.231883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:06.231883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:06.231883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:06.231883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:06.231883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:06.231883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:06.231883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 05:17:06.231883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:06.231883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 309/636] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:17:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:06.239090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2688e-ef84-4f88-b827-34bc8710890d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:06.239090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2688e-ef84-4f88-b827-34bc8710890d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:06.239090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2688e-ef84-4f88-b827-34bc8710890d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:06.266175 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-fdd2688e-ef84-4f88-b827-34bc8710890d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:06.267472 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fdd2688e-ef84-4f88-b827-34bc8710890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:06.309842 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109206e0-6ef2-41b7-b22f-3ce0bdb57eca None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:06.319432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:06.320200 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:06.351874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a907a0-da3c-496b-a71a-beb71a195c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:06.352565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a907a0-da3c-496b-a71a-beb71a195c9b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:06.352848 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a907a0-da3c-496b-a71a-beb71a195c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:06.353121 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a907a0-da3c-496b-a71a-beb71a195c9b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:06.362797 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a907a0-da3c-496b-a71a-beb71a195c9b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:06.364626 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a907a0-da3c-496b-a71a-beb71a195c9b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:06.370044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 310/637] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:17:06.371304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2688e-ef84-4f88-b827-34bc8710890d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-b0a907a0-da3c-496b-a71a-beb71a195c9b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:06.389202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:06.390198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:06.391124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:06.391124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:06.391124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:06.391124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:06.391124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:06.391124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:06.391124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:06.391124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:06.391124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:17:06.391124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:06.401764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 328/638] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 05:17:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:06.402773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a907a0-da3c-496b-a71a-beb71a195c9b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:06.565083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2688e-ef84-4f88-b827-34bc8710890d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:06.591523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a907a0-da3c-496b-a71a-beb71a195c9b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:06.628474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cf4bd-3943-400f-a311-8a731050b239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:06.629710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cf4bd-3943-400f-a311-8a731050b239 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:06.631300 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cf4bd-3943-400f-a311-8a731050b239 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:06.631784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cf4bd-3943-400f-a311-8a731050b239 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:06.632748 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cf4bd-3943-400f-a311-8a731050b239 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:06.633472 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cf4bd-3943-400f-a311-8a731050b239 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:06.636105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cf4bd-3943-400f-a311-8a731050b239 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:06.659594 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757a880-241b-49c4-866a-d205fe43e7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:06.660307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757a880-241b-49c4-866a-d205fe43e7ef None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:06.660935 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757a880-241b-49c4-866a-d205fe43e7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:06.661595 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757a880-241b-49c4-866a-d205fe43e7ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:06.662704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 311/639] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 11 05:17:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:06.663849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cf4bd-3943-400f-a311-8a731050b239 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:06.676825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757a880-241b-49c4-866a-d205fe43e7ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5757a880-241b-49c4-866a-d205fe43e7ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:06.685820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:06.686748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:06.687580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:06.687580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:06.687580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:06.687580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:06.687580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:06.687580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:06.687580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:06.687580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:06.687580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:06.687580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:06.687580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:06.687580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:06.692225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 329/640] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:17:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:06.693080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757a880-241b-49c4-866a-d205fe43e7ef None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:06.700539 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:06.732485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:06.790148 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cf4bd-3943-400f-a311-8a731050b239 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:06.810999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757a880-241b-49c4-866a-d205fe43e7ef None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:06.844404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d4b22-dbf5-43fb-8437-fa8627667c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:06.845242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d4b22-dbf5-43fb-8437-fa8627667c97 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:06.845541 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d4b22-dbf5-43fb-8437-fa8627667c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:06.846468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d4b22-dbf5-43fb-8437-fa8627667c97 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:06.847118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6df7-cd3f-40fb-bddc-e9c91598b8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:06.847392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d4b22-dbf5-43fb-8437-fa8627667c97 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:06.847700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d4b22-dbf5-43fb-8437-fa8627667c97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:06.847795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6df7-cd3f-40fb-bddc-e9c91598b8cc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:06.848024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6df7-cd3f-40fb-bddc-e9c91598b8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:06.848317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6df7-cd3f-40fb-bddc-e9c91598b8cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:06.849364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d4b22-dbf5-43fb-8437-fa8627667c97 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:06.859633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a6df7-cd3f-40fb-bddc-e9c91598b8cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-265a6df7-cd3f-40fb-bddc-e9c91598b8cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:06.865152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:06.866139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 330/641] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:17:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:06.866996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6df7-cd3f-40fb-bddc-e9c91598b8cc None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:06.874517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 312/642] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 05:17:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:06.874517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d4b22-dbf5-43fb-8437-fa8627667c97 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:06.959051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:06.973123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:06.980779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d4b22-dbf5-43fb-8437-fa8627667c97 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:06.987384 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6df7-cd3f-40fb-bddc-e9c91598b8cc None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:07.018149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c762d-3ccb-41a6-9efb-894d9b07b457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:07.018753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c762d-3ccb-41a6-9efb-894d9b07b457 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:07.019035 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c762d-3ccb-41a6-9efb-894d9b07b457 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:07.019313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c762d-3ccb-41a6-9efb-894d9b07b457 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:07.020104 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c762d-3ccb-41a6-9efb-894d9b07b457 None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:07.020508 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c762d-3ccb-41a6-9efb-894d9b07b457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:07.021999 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c762d-3ccb-41a6-9efb-894d9b07b457 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:07.022813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83116d-d9c3-485c-999f-0f4914c644d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:07.023422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83116d-d9c3-485c-999f-0f4914c644d1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:07.023700 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83116d-d9c3-485c-999f-0f4914c644d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:07.023972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83116d-d9c3-485c-999f-0f4914c644d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:07.030649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 313/643] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:17:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:07.031252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c762d-3ccb-41a6-9efb-894d9b07b457 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:07.101442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83116d-d9c3-485c-999f-0f4914c644d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:07.102825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83116d-d9c3-485c-999f-0f4914c644d1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:07.140532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c762d-3ccb-41a6-9efb-894d9b07b457 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:07.179174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52c172-63da-4b0b-bafa-62b22b032f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:07.179788 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52c172-63da-4b0b-bafa-62b22b032f36 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:07.180341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52c172-63da-4b0b-bafa-62b22b032f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:07.180483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52c172-63da-4b0b-bafa-62b22b032f36 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:07.181124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52c172-63da-4b0b-bafa-62b22b032f36 None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:07.181434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52c172-63da-4b0b-bafa-62b22b032f36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:07.182810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52c172-63da-4b0b-bafa-62b22b032f36 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:07.193879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 314/644] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:17:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:07.195224 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:07.195773 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52c172-63da-4b0b-bafa-62b22b032f36 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:07.205151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:07.218707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 331/645] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:17:07.218707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83116d-d9c3-485c-999f-0f4914c644d1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:07.301716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52c172-63da-4b0b-bafa-62b22b032f36 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:07.326467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83116d-d9c3-485c-999f-0f4914c644d1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:07.339769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bac3e7-3279-4e4d-98b7-3120cef1a9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:07.340473 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bac3e7-3279-4e4d-98b7-3120cef1a9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:07.340754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bac3e7-3279-4e4d-98b7-3120cef1a9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:07.341030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bac3e7-3279-4e4d-98b7-3120cef1a9b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:07.367158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4db377-5558-4ca4-9a8c-fbdcdb0f360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:07.367715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4db377-5558-4ca4-9a8c-fbdcdb0f360b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:07.368015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4db377-5558-4ca4-9a8c-fbdcdb0f360b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:07.368305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4db377-5558-4ca4-9a8c-fbdcdb0f360b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:07.379030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4db377-5558-4ca4-9a8c-fbdcdb0f360b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:07.380510 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4db377-5558-4ca4-9a8c-fbdcdb0f360b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-cb4db377-5558-4ca4-9a8c-fbdcdb0f360b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:07.395945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:07.397011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:07.398307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:07.398307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:07.398307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:07.398307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 332/646] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:07.398307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4db377-5558-4ca4-9a8c-fbdcdb0f360b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:07.418411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bac3e7-3279-4e4d-98b7-3120cef1a9b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:07.420141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bac3e7-3279-4e4d-98b7-3120cef1a9b4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:07.428791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:07.446025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:07.508284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4db377-5558-4ca4-9a8c-fbdcdb0f360b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:07.522597 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 315/647] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:17:07.523429 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bac3e7-3279-4e4d-98b7-3120cef1a9b4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:07.546347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223b827-ed20-4a5d-807a-aa2a322b7c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:07.546967 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223b827-ed20-4a5d-807a-aa2a322b7c48 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:07.547243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223b827-ed20-4a5d-807a-aa2a322b7c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:07.547507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223b827-ed20-4a5d-807a-aa2a322b7c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:07.558294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223b827-ed20-4a5d-807a-aa2a322b7c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:07.559719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223b827-ed20-4a5d-807a-aa2a322b7c48 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-8223b827-ed20-4a5d-807a-aa2a322b7c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:07.573339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:07.574461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:07.575355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:07.575355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:07.575355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:07.575355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 333/648] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:07.575578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223b827-ed20-4a5d-807a-aa2a322b7c48 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:07.623288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bac3e7-3279-4e4d-98b7-3120cef1a9b4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:07.631725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:07.658783 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa7452-f14e-4f01-ab53-cf4d3de3e4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:07.659403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa7452-f14e-4f01-ab53-cf4d3de3e4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:07.659678 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa7452-f14e-4f01-ab53-cf4d3de3e4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:07.659943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa7452-f14e-4f01-ab53-cf4d3de3e4a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:07.665294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:07.669732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa7452-f14e-4f01-ab53-cf4d3de3e4a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:07.671167 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa7452-f14e-4f01-ab53-cf4d3de3e4a2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:07.675875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223b827-ed20-4a5d-807a-aa2a322b7c48 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-9baa7452-f14e-4f01-ab53-cf4d3de3e4a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:07.686899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:07.688076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:07.689957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:07.689957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:07.689957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:07.689957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 316/649] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:07.689957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb7d97-d311-4cb6-a387-72095e5040e2 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:07.689957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb7d97-d311-4cb6-a387-72095e5040e2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:07.690443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb7d97-d311-4cb6-a387-72095e5040e2 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:07.695936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 317/650] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:07.695936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb7d97-d311-4cb6-a387-72095e5040e2 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:07.716783 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9e1a3-7f54-43ee-878e-f2acdb521ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:07.717606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9e1a3-7f54-43ee-878e-f2acdb521ded None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:07.717889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9e1a3-7f54-43ee-878e-f2acdb521ded None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:07.718151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9e1a3-7f54-43ee-878e-f2acdb521ded None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:07.718782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9e1a3-7f54-43ee-878e-f2acdb521ded None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:07.719079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9e1a3-7f54-43ee-878e-f2acdb521ded None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:07.720466 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9e1a3-7f54-43ee-878e-f2acdb521ded None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:07.732354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 334/651] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:07.733207 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9e1a3-7f54-43ee-878e-f2acdb521ded None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:07.798003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb7d97-d311-4cb6-a387-72095e5040e2 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:07.836008 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9e1a3-7f54-43ee-878e-f2acdb521ded None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:07.837861 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c6bb2-1f9b-4e52-af98-5252a6441c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:07.838485 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c6bb2-1f9b-4e52-af98-5252a6441c53 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:07.838759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c6bb2-1f9b-4e52-af98-5252a6441c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:07.839025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c6bb2-1f9b-4e52-af98-5252a6441c53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:07.849493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c6bb2-1f9b-4e52-af98-5252a6441c53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:07.850923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c6bb2-1f9b-4e52-af98-5252a6441c53 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-475c6bb2-1f9b-4e52-af98-5252a6441c53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:07.865412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:07.866452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:07.868066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:07.868066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:07.868066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:07.868066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 318/652] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:07.868066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c6bb2-1f9b-4e52-af98-5252a6441c53 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:07.870349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8ab0a-fab8-40e9-8d76-bd7123c2845a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:07.870951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8ab0a-fab8-40e9-8d76-bd7123c2845a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:07.871226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8ab0a-fab8-40e9-8d76-bd7123c2845a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:07.871486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8ab0a-fab8-40e9-8d76-bd7123c2845a None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:07.872145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8ab0a-fab8-40e9-8d76-bd7123c2845a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:07.872443 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8ab0a-fab8-40e9-8d76-bd7123c2845a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:07.873830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8ab0a-fab8-40e9-8d76-bd7123c2845a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:07.883269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:07.886738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 335/653] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:07.888333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f3d7e-7da9-4315-844d-c53a83283133 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:07.888333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f3d7e-7da9-4315-844d-c53a83283133 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:07.888784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f3d7e-7da9-4315-844d-c53a83283133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:07.906776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-aa8f3d7e-7da9-4315-844d-c53a83283133 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:07.907967 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-aa8f3d7e-7da9-4315-844d-c53a83283133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:07.915078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:07.974961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c6bb2-1f9b-4e52-af98-5252a6441c53 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:07.976190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 336/654] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:17:07.977018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f3d7e-7da9-4315-844d-c53a83283133 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:08.026952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec650d-8a1b-44e2-88f8-18b0919f4e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:08.027559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec650d-8a1b-44e2-88f8-18b0919f4e2d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:08.027834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec650d-8a1b-44e2-88f8-18b0919f4e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:08.028136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec650d-8a1b-44e2-88f8-18b0919f4e2d None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:08.028773 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec650d-8a1b-44e2-88f8-18b0919f4e2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:08.029075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec650d-8a1b-44e2-88f8-18b0919f4e2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:08.030437 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec650d-8a1b-44e2-88f8-18b0919f4e2d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:08.043650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 319/655] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:08.044270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec650d-8a1b-44e2-88f8-18b0919f4e2d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:08.083629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f3d7e-7da9-4315-844d-c53a83283133 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:08.118351 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a08247-cdd6-49ed-ae45-6ad11c0563b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:08.119069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a08247-cdd6-49ed-ae45-6ad11c0563b6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:08.119347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a08247-cdd6-49ed-ae45-6ad11c0563b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:08.119615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a08247-cdd6-49ed-ae45-6ad11c0563b6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:08.122694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:08.126916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a08247-cdd6-49ed-ae45-6ad11c0563b6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:08.128924 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a08247-cdd6-49ed-ae45-6ad11c0563b6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-29a08247-cdd6-49ed-ae45-6ad11c0563b6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:08.143032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:08.144040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:08.145036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:08.145036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:08.145036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:08.145036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:08.145036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:08.145036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:08.145036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:08.145036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:08.145036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 05:17:08.145036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:08.145036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 337/656] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:17:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:08.145036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a08247-cdd6-49ed-ae45-6ad11c0563b6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:08.148684 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:08.165992 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec650d-8a1b-44e2-88f8-18b0919f4e2d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:08.202649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d0879-a9fa-4819-963d-1eba7675e133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:08.203260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d0879-a9fa-4819-963d-1eba7675e133 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:08.203545 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d0879-a9fa-4819-963d-1eba7675e133 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:08.203822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d0879-a9fa-4819-963d-1eba7675e133 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:08.204491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d0879-a9fa-4819-963d-1eba7675e133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:08.204796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d0879-a9fa-4819-963d-1eba7675e133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:08.206865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d0879-a9fa-4819-963d-1eba7675e133 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:08.218150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 320/657] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:08.218940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d0879-a9fa-4819-963d-1eba7675e133 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:08.258374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a08247-cdd6-49ed-ae45-6ad11c0563b6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:08.291942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4837e0-2e91-4bc6-b3ce-7a8b5ac7ccff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:08.292597 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4837e0-2e91-4bc6-b3ce-7a8b5ac7ccff None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:08.292878 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4837e0-2e91-4bc6-b3ce-7a8b5ac7ccff None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:08.293147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4837e0-2e91-4bc6-b3ce-7a8b5ac7ccff None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:08.300664 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4837e0-2e91-4bc6-b3ce-7a8b5ac7ccff None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:08.302186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4837e0-2e91-4bc6-b3ce-7a8b5ac7ccff None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-bb4837e0-2e91-4bc6-b3ce-7a8b5ac7ccff None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:08.313182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:08.314359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:08.315801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:08.315801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:08.315801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:08.315801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:08.315801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:08.315801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:08.315801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:08.315801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:08.315801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 05:17:08.315801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:08.315801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 338/658] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:17:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:08.315801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4837e0-2e91-4bc6-b3ce-7a8b5ac7ccff None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:08.323702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d0879-a9fa-4819-963d-1eba7675e133 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:08.352703 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:08.358969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1fd8b-dd29-419a-8985-1b572d65776d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:08.359609 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1fd8b-dd29-419a-8985-1b572d65776d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:08.359888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1fd8b-dd29-419a-8985-1b572d65776d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:08.360194 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1fd8b-dd29-419a-8985-1b572d65776d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:08.367246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1fd8b-dd29-419a-8985-1b572d65776d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:08.368739 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1fd8b-dd29-419a-8985-1b572d65776d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:08.375512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-b8d1fd8b-dd29-419a-8985-1b572d65776d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:08.381550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:08.382617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:08.383910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:08.383910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:08.383910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:08.383910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:08.383910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:08.383910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:08.383910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:08.383910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:08.383910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 321/659] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:08.383910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1fd8b-dd29-419a-8985-1b572d65776d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:08.417995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4837e0-2e91-4bc6-b3ce-7a8b5ac7ccff None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:08.452387 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39123239-b3b5-4aea-bedc-6457e49c0f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:08.452989 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39123239-b3b5-4aea-bedc-6457e49c0f14 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:08.453267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39123239-b3b5-4aea-bedc-6457e49c0f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:08.453540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39123239-b3b5-4aea-bedc-6457e49c0f14 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:08.454259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39123239-b3b5-4aea-bedc-6457e49c0f14 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:08.454563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39123239-b3b5-4aea-bedc-6457e49c0f14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:08.456260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39123239-b3b5-4aea-bedc-6457e49c0f14 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:08.473515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 339/660] 10.222.0.33 () {56 vars in 1137 bytes} [Sun Sep 11 05:17:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:08.474303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39123239-b3b5-4aea-bedc-6457e49c0f14 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:08.487603 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1fd8b-dd29-419a-8985-1b572d65776d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:08.520700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddf8ba-850e-4b81-ae54-43fc54b6f90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:08.521317 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf8ba-850e-4b81-ae54-43fc54b6f90b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:08.521601 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf8ba-850e-4b81-ae54-43fc54b6f90b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:08.521872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf8ba-850e-4b81-ae54-43fc54b6f90b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:08.528403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddf8ba-850e-4b81-ae54-43fc54b6f90b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:08.529839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddf8ba-850e-4b81-ae54-43fc54b6f90b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ddddf8ba-850e-4b81-ae54-43fc54b6f90b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:08.541446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:08.542319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:08.543366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:08.543366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:08.543366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:08.543366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:08.543366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:08.543366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:08.543366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:08.543366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:08.543366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 322/661] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:08.543366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddf8ba-850e-4b81-ae54-43fc54b6f90b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:08.573931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39123239-b3b5-4aea-bedc-6457e49c0f14 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:08.591518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:08.593918 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:08.608730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085923f-e08e-4cf5-b511-abd428dbe5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:08.609362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085923f-e08e-4cf5-b511-abd428dbe5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:08.609643 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085923f-e08e-4cf5-b511-abd428dbe5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:08.609916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085923f-e08e-4cf5-b511-abd428dbe5a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:08.620264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085923f-e08e-4cf5-b511-abd428dbe5a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-f085923f-e08e-4cf5-b511-abd428dbe5a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:08.625028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:08.626141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 340/662] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:17:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:08.627751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085923f-e08e-4cf5-b511-abd428dbe5a0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:08.645328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddf8ba-850e-4b81-ae54-43fc54b6f90b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:08.679284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f91807-f317-4aba-ab2b-8e4aaec7ac2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:08.679885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f91807-f317-4aba-ab2b-8e4aaec7ac2a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:08.680200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f91807-f317-4aba-ab2b-8e4aaec7ac2a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:08.680475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f91807-f317-4aba-ab2b-8e4aaec7ac2a None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:08.681195 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f91807-f317-4aba-ab2b-8e4aaec7ac2a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:08.681518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f91807-f317-4aba-ab2b-8e4aaec7ac2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:08.682982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f91807-f317-4aba-ab2b-8e4aaec7ac2a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:08.699798 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 323/663] 10.222.0.33 () {56 vars in 1139 bytes} [Sun Sep 11 05:17:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:08.700629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f91807-f317-4aba-ab2b-8e4aaec7ac2a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:08.726111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085923f-e08e-4cf5-b511-abd428dbe5a0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:08.760764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9a47f-7383-491e-8f07-c8c08fd29398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:08.761381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9a47f-7383-491e-8f07-c8c08fd29398 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:08.761657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9a47f-7383-491e-8f07-c8c08fd29398 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:08.761926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9a47f-7383-491e-8f07-c8c08fd29398 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:08.762643 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9a47f-7383-491e-8f07-c8c08fd29398 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:08.762942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9a47f-7383-491e-8f07-c8c08fd29398 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:08.764396 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9a47f-7383-491e-8f07-c8c08fd29398 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:08.775853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 341/664] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:08.776671 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9a47f-7383-491e-8f07-c8c08fd29398 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:08.802987 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f91807-f317-4aba-ab2b-8e4aaec7ac2a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:08.812506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:08.823734 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:08.839140 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b52cc1-d838-479f-9484-312782e4b4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:08.839744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b52cc1-d838-479f-9484-312782e4b4bc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:08.840046 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b52cc1-d838-479f-9484-312782e4b4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:08.840316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b52cc1-d838-479f-9484-312782e4b4bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:08.882327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9a47f-7383-491e-8f07-c8c08fd29398 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:08.907694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b52cc1-d838-479f-9484-312782e4b4bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:08.909356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b52cc1-d838-479f-9484-312782e4b4bc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:08.915499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3982b-bd18-4b59-a351-383f2715a922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:08.916130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3982b-bd18-4b59-a351-383f2715a922 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:08.916409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3982b-bd18-4b59-a351-383f2715a922 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:08.916676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3982b-bd18-4b59-a351-383f2715a922 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:08.917684 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3982b-bd18-4b59-a351-383f2715a922 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:08.917996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3982b-bd18-4b59-a351-383f2715a922 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:08.919409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3982b-bd18-4b59-a351-383f2715a922 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:08.932177 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 342/665] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:08.932992 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3982b-bd18-4b59-a351-383f2715a922 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:09.013013 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 324/666] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:17:09.013943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b52cc1-d838-479f-9484-312782e4b4bc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:09.024547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:09.031939 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:09.036015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3982b-bd18-4b59-a351-383f2715a922 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:09.070180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8cb1e-933f-475e-b6f8-118bea29a030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:09.070791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8cb1e-933f-475e-b6f8-118bea29a030 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:09.071065 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8cb1e-933f-475e-b6f8-118bea29a030 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:09.071337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8cb1e-933f-475e-b6f8-118bea29a030 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:09.072019 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8cb1e-933f-475e-b6f8-118bea29a030 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:09.072319 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8cb1e-933f-475e-b6f8-118bea29a030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:09.073674 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8cb1e-933f-475e-b6f8-118bea29a030 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:09.082220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 343/667] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:17:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:09.083137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8cb1e-933f-475e-b6f8-118bea29a030 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:09.117872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b52cc1-d838-479f-9484-312782e4b4bc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:09.151694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487b44f-f0b9-4d10-8d1d-5e8e65f2e2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:09.152332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487b44f-f0b9-4d10-8d1d-5e8e65f2e2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:09.152612 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487b44f-f0b9-4d10-8d1d-5e8e65f2e2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:09.152883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487b44f-f0b9-4d10-8d1d-5e8e65f2e2f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:09.181467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8cb1e-933f-475e-b6f8-118bea29a030 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:09.217304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62880c87-1c4a-42bd-ab4d-54d4ef3c4c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:09.217937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62880c87-1c4a-42bd-ab4d-54d4ef3c4c63 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:09.218217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62880c87-1c4a-42bd-ab4d-54d4ef3c4c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:09.218489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62880c87-1c4a-42bd-ab4d-54d4ef3c4c63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:09.219209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62880c87-1c4a-42bd-ab4d-54d4ef3c4c63 None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=0330c79c391d463b91f99e030c6dcc07) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:09.219523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62880c87-1c4a-42bd-ab4d-54d4ef3c4c63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:09.219702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487b44f-f0b9-4d10-8d1d-5e8e65f2e2f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:09.221181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62880c87-1c4a-42bd-ab4d-54d4ef3c4c63 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:09.221328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487b44f-f0b9-4d10-8d1d-5e8e65f2e2f6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:09.226787 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 344/668] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:17:09 2022] GET /identity/v3/role_assignments?user.id=0330c79c391d463b91f99e030c6dcc07&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:09.227570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62880c87-1c4a-42bd-ab4d-54d4ef3c4c63 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:09.230998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:09.271331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:09.341816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 325/669] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:17:09.342932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487b44f-f0b9-4d10-8d1d-5e8e65f2e2f6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:09.375256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62880c87-1c4a-42bd-ab4d-54d4ef3c4c63 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:09.416245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d998a-f774-4e1e-aafb-6df248be5523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:09.416858 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d998a-f774-4e1e-aafb-6df248be5523 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:09.417143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d998a-f774-4e1e-aafb-6df248be5523 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:09.417420 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d998a-f774-4e1e-aafb-6df248be5523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:09.418146 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d998a-f774-4e1e-aafb-6df248be5523 None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=3bd5a9a09f034c05815c631a44d1cdd9) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:09.418455 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d998a-f774-4e1e-aafb-6df248be5523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:09.419892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d998a-f774-4e1e-aafb-6df248be5523 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:09.426912 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 345/670] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:17:09 2022] GET /identity/v3/role_assignments?user.id=3bd5a9a09f034c05815c631a44d1cdd9&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:09.427738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d998a-f774-4e1e-aafb-6df248be5523 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:09.462896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487b44f-f0b9-4d10-8d1d-5e8e65f2e2f6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:09.478936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:09.519669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fed2f-b3a9-43ae-bdaf-9b0f0669abae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:09.520384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fed2f-b3a9-43ae-bdaf-9b0f0669abae None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:09.520662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fed2f-b3a9-43ae-bdaf-9b0f0669abae None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:09.520936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fed2f-b3a9-43ae-bdaf-9b0f0669abae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:09.533780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fed2f-b3a9-43ae-bdaf-9b0f0669abae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:09.535616 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fed2f-b3a9-43ae-bdaf-9b0f0669abae None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:09.553759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-796fed2f-b3a9-43ae-bdaf-9b0f0669abae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:09.558964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:09.559937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:09.560906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:09.560906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:09.560906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:09.562312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 326/671] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:09.567636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fed2f-b3a9-43ae-bdaf-9b0f0669abae None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:09.567873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d998a-f774-4e1e-aafb-6df248be5523 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:09.610697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf531e4-26b5-4078-b501-55108a8e770f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:09.611857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf531e4-26b5-4078-b501-55108a8e770f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:09.612985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf531e4-26b5-4078-b501-55108a8e770f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:09.612985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf531e4-26b5-4078-b501-55108a8e770f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:09.625292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf531e4-26b5-4078-b501-55108a8e770f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:09.626841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf531e4-26b5-4078-b501-55108a8e770f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-ccf531e4-26b5-4078-b501-55108a8e770f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:09.644679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:09.645675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:09.646627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:09.646627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:09.646627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:09.646627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 346/672] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:09.646871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf531e4-26b5-4078-b501-55108a8e770f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:09.710643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fed2f-b3a9-43ae-bdaf-9b0f0669abae None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:09.752844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260017dd-a187-4f45-a81e-74ef78ba8788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:09.753797 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260017dd-a187-4f45-a81e-74ef78ba8788 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:09.754087 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260017dd-a187-4f45-a81e-74ef78ba8788 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:09.754364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260017dd-a187-4f45-a81e-74ef78ba8788 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:09.765566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260017dd-a187-4f45-a81e-74ef78ba8788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:09.767039 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260017dd-a187-4f45-a81e-74ef78ba8788 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:09.779896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf531e4-26b5-4078-b501-55108a8e770f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-260017dd-a187-4f45-a81e-74ef78ba8788 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:09.784026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:09.785101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:09.786233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:09.786233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:09.786233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:09.786233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 327/673] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:09.786489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260017dd-a187-4f45-a81e-74ef78ba8788 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:09.807794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:09.831696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959a98-47a0-49c8-b787-1b4fdc9cca4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:09.832341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959a98-47a0-49c8-b787-1b4fdc9cca4e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:09.832623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959a98-47a0-49c8-b787-1b4fdc9cca4e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:09.832898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959a98-47a0-49c8-b787-1b4fdc9cca4e None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:09.833549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959a98-47a0-49c8-b787-1b4fdc9cca4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:09.833851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959a98-47a0-49c8-b787-1b4fdc9cca4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:09.835296 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959a98-47a0-49c8-b787-1b4fdc9cca4e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:09.848867 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 347/674] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:09.849794 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959a98-47a0-49c8-b787-1b4fdc9cca4e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:09.852633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:09.931484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260017dd-a187-4f45-a81e-74ef78ba8788 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:09.972817 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89260e-5dfd-47ca-8b6c-355f8334e84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:09.973891 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89260e-5dfd-47ca-8b6c-355f8334e84b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:09.974432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89260e-5dfd-47ca-8b6c-355f8334e84b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:09.974913 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89260e-5dfd-47ca-8b6c-355f8334e84b None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:09.975033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959a98-47a0-49c8-b787-1b4fdc9cca4e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:09.975881 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89260e-5dfd-47ca-8b6c-355f8334e84b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:09.976709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89260e-5dfd-47ca-8b6c-355f8334e84b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:09.979262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89260e-5dfd-47ca-8b6c-355f8334e84b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:10.000885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 328/675] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:10.003196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89260e-5dfd-47ca-8b6c-355f8334e84b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:10.016797 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3f4e8-5c2f-4dd6-b86f-ef7ce442788e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:10.018015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3f4e8-5c2f-4dd6-b86f-ef7ce442788e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:10.018308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3f4e8-5c2f-4dd6-b86f-ef7ce442788e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:10.018578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3f4e8-5c2f-4dd6-b86f-ef7ce442788e None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:10.019225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3f4e8-5c2f-4dd6-b86f-ef7ce442788e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:10.019534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3f4e8-5c2f-4dd6-b86f-ef7ce442788e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:10.020940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3f4e8-5c2f-4dd6-b86f-ef7ce442788e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:10.034526 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 348/676] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:10.035768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3f4e8-5c2f-4dd6-b86f-ef7ce442788e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:10.105356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:10.123738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:10.152787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89260e-5dfd-47ca-8b6c-355f8334e84b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:10.167137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3f4e8-5c2f-4dd6-b86f-ef7ce442788e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:10.202505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46407da-a9f1-43ec-a10c-e03c917d7a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:10.202738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607f69c-c353-49d1-a932-6c26d4342e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:10.203241 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607f69c-c353-49d1-a932-6c26d4342e4b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:10.203535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607f69c-c353-49d1-a932-6c26d4342e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:10.203799 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607f69c-c353-49d1-a932-6c26d4342e4b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:10.204623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46407da-a9f1-43ec-a10c-e03c917d7a0c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:10.205193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46407da-a9f1-43ec-a10c-e03c917d7a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:10.205773 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46407da-a9f1-43ec-a10c-e03c917d7a0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:10.212240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607f69c-c353-49d1-a932-6c26d4342e4b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:10.213688 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607f69c-c353-49d1-a932-6c26d4342e4b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:10.215203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46407da-a9f1-43ec-a10c-e03c917d7a0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:10.217713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46407da-a9f1-43ec-a10c-e03c917d7a0c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-7607f69c-c353-49d1-a932-6c26d4342e4b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:10.226173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:10.227164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:10.228201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:10.228201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:10.228201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:10.228201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:10.228201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:10.228201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:10.228201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:10.228201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:10.228201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 05:17:10.228201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:10.231967 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 349/677] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:17:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:10.232835 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607f69c-c353-49d1-a932-6c26d4342e4b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-b46407da-a9f1-43ec-a10c-e03c917d7a0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:10.234293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:10.235190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:10.236141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:10.236141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:10.236141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:10.236141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:10.236141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:10.236141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:10.236141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:10.236141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:10.238944 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 329/678] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:10.239500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46407da-a9f1-43ec-a10c-e03c917d7a0c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:10.346422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607f69c-c353-49d1-a932-6c26d4342e4b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:10.369601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:10.379838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46407da-a9f1-43ec-a10c-e03c917d7a0c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:10.382587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63821bfc-c103-4de3-b678-36113ec92548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:10.383696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63821bfc-c103-4de3-b678-36113ec92548 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:10.383696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63821bfc-c103-4de3-b678-36113ec92548 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:10.383872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63821bfc-c103-4de3-b678-36113ec92548 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:10.390612 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63821bfc-c103-4de3-b678-36113ec92548 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:10.392089 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63821bfc-c103-4de3-b678-36113ec92548 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-63821bfc-c103-4de3-b678-36113ec92548 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:10.404729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:10.405632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:10.406576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:10.406576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:10.406576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:10.406576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:10.406576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:10.406576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:10.406576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:10.406576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:10.406576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 350/679] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:10.407156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63821bfc-c103-4de3-b678-36113ec92548 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:10.418036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:10.425906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da06b82-58ba-424f-b43e-6c65ccb77ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:10.426862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da06b82-58ba-424f-b43e-6c65ccb77ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:10.427346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da06b82-58ba-424f-b43e-6c65ccb77ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:10.427846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da06b82-58ba-424f-b43e-6c65ccb77ec9 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:10.428818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da06b82-58ba-424f-b43e-6c65ccb77ec9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:10.429423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da06b82-58ba-424f-b43e-6c65ccb77ec9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:10.431063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da06b82-58ba-424f-b43e-6c65ccb77ec9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:10.451072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 330/680] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 11 05:17:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:10.452328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da06b82-58ba-424f-b43e-6c65ccb77ec9 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:10.517981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63821bfc-c103-4de3-b678-36113ec92548 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:10.552733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fba8f-2c73-48f6-afe2-0853b8a039c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:10.553345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fba8f-2c73-48f6-afe2-0853b8a039c6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:10.553634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fba8f-2c73-48f6-afe2-0853b8a039c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:10.553909 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fba8f-2c73-48f6-afe2-0853b8a039c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:10.554602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fba8f-2c73-48f6-afe2-0853b8a039c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:10.554910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fba8f-2c73-48f6-afe2-0853b8a039c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:10.556621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fba8f-2c73-48f6-afe2-0853b8a039c6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:10.569684 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 351/681] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:10.570161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da06b82-58ba-424f-b43e-6c65ccb77ec9 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:10.570571 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fba8f-2c73-48f6-afe2-0853b8a039c6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:10.609805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35075d06-85e9-4a3c-99bf-650d42c65bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:10.610857 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35075d06-85e9-4a3c-99bf-650d42c65bec None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:10.611866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35075d06-85e9-4a3c-99bf-650d42c65bec None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:10.612446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35075d06-85e9-4a3c-99bf-650d42c65bec None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:10.613538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35075d06-85e9-4a3c-99bf-650d42c65bec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:10.614062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35075d06-85e9-4a3c-99bf-650d42c65bec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:10.614742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:10.616795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35075d06-85e9-4a3c-99bf-650d42c65bec None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:10.630947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 331/682] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:10.632334 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35075d06-85e9-4a3c-99bf-650d42c65bec None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:10.674632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fba8f-2c73-48f6-afe2-0853b8a039c6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:10.690033 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:10.709817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52f6c3-4e23-447b-8f7e-c86e6887f132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:10.710427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52f6c3-4e23-447b-8f7e-c86e6887f132 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:10.710707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52f6c3-4e23-447b-8f7e-c86e6887f132 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:10.710981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52f6c3-4e23-447b-8f7e-c86e6887f132 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:10.750261 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35075d06-85e9-4a3c-99bf-650d42c65bec None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:10.782370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52f6c3-4e23-447b-8f7e-c86e6887f132 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:10.784495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52f6c3-4e23-447b-8f7e-c86e6887f132 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:10.797342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0b113-5497-4cd9-baaf-451898e840ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:10.798378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0b113-5497-4cd9-baaf-451898e840ca None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:10.804940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0b113-5497-4cd9-baaf-451898e840ca None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:10.805622 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0b113-5497-4cd9-baaf-451898e840ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:10.806610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0b113-5497-4cd9-baaf-451898e840ca None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=8ef93155a40f4c69b320ca962d4e8868) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:10.807128 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0b113-5497-4cd9-baaf-451898e840ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:10.809028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0b113-5497-4cd9-baaf-451898e840ca None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:10.817415 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 332/683] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:17:10 2022] GET /identity/v3/role_assignments?user.id=8ef93155a40f4c69b320ca962d4e8868&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:10.818575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0b113-5497-4cd9-baaf-451898e840ca None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:10.828494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:10.897329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 352/684] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:17:10.898570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52f6c3-4e23-447b-8f7e-c86e6887f132 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:10.942706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0b113-5497-4cd9-baaf-451898e840ca None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:10.959670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:10.984760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed338b9f-67ad-4fdf-8294-c855e1862a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:10.985761 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed338b9f-67ad-4fdf-8294-c855e1862a92 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:10.986309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed338b9f-67ad-4fdf-8294-c855e1862a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:10.986788 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed338b9f-67ad-4fdf-8294-c855e1862a92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:10.987879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed338b9f-67ad-4fdf-8294-c855e1862a92 None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=0bf2a5de0e744505a1317ce9fa224643) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:10.988592 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed338b9f-67ad-4fdf-8294-c855e1862a92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:10.990278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed338b9f-67ad-4fdf-8294-c855e1862a92 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:10.998216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 333/685] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:17:10 2022] GET /identity/v3/role_assignments?user.id=0bf2a5de0e744505a1317ce9fa224643&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:11.009134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52f6c3-4e23-447b-8f7e-c86e6887f132 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:11.053607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4806d8-0328-41f1-b550-168f333fb46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:11.054289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4806d8-0328-41f1-b550-168f333fb46a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:11.054575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4806d8-0328-41f1-b550-168f333fb46a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:11.054853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4806d8-0328-41f1-b550-168f333fb46a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:11.070711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4806d8-0328-41f1-b550-168f333fb46a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:11.072232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4806d8-0328-41f1-b550-168f333fb46a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:11.074281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-0b4806d8-0328-41f1-b550-168f333fb46a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:11.090277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:11.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:11.092266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:11.092266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:11.092266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:11.096364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 353/686] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:11.099367 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed338b9f-67ad-4fdf-8294-c855e1862a92 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:11.223846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed338b9f-67ad-4fdf-8294-c855e1862a92 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:11.261224 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa3ae8-f4ad-4159-88ae-aad1a5cac760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:11.262133 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa3ae8-f4ad-4159-88ae-aad1a5cac760 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:11.262417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa3ae8-f4ad-4159-88ae-aad1a5cac760 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:11.262693 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa3ae8-f4ad-4159-88ae-aad1a5cac760 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:11.263387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa3ae8-f4ad-4159-88ae-aad1a5cac760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:11.263645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa3ae8-f4ad-4159-88ae-aad1a5cac760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:11.265061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa3ae8-f4ad-4159-88ae-aad1a5cac760 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:11.279647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 334/687] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:11.285212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4806d8-0328-41f1-b550-168f333fb46a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:11.409454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4806d8-0328-41f1-b550-168f333fb46a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:11.451538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c6590-62a4-43b4-b431-50095119d472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:11.452212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c6590-62a4-43b4-b431-50095119d472 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:11.452495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c6590-62a4-43b4-b431-50095119d472 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:11.452765 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c6590-62a4-43b4-b431-50095119d472 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:11.460712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c6590-62a4-43b4-b431-50095119d472 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:11.462175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c6590-62a4-43b4-b431-50095119d472 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5a0c6590-62a4-43b4-b431-50095119d472 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:11.474813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:11.475719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:11.476641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:11.476641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:11.476641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:11.476641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:11.476641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:11.476641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:11.476641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:11.476641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:11.476641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 354/688] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:11.485096 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa3ae8-f4ad-4159-88ae-aad1a5cac760 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:11.555065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:11.646712 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa3ae8-f4ad-4159-88ae-aad1a5cac760 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:11.699705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ca8f6-b1b3-426d-9e14-fb28f6671d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:11.700788 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ca8f6-b1b3-426d-9e14-fb28f6671d84 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:11.701073 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ca8f6-b1b3-426d-9e14-fb28f6671d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:11.701346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ca8f6-b1b3-426d-9e14-fb28f6671d84 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:11.702082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ca8f6-b1b3-426d-9e14-fb28f6671d84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:11.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ca8f6-b1b3-426d-9e14-fb28f6671d84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:11.703897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ca8f6-b1b3-426d-9e14-fb28f6671d84 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:11.718554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 335/689] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:11.724491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c6590-62a4-43b4-b431-50095119d472 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:11.818136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f729383-7c1c-4ab5-aa8f-90bcad96397a None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:11.818414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f729383-7c1c-4ab5-aa8f-90bcad96397a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:11.818679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f729383-7c1c-4ab5-aa8f-90bcad96397a None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:11.820022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 336/690] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:11.825031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa8532-4010-44d1-a88f-a891cf347058 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:11.825317 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa8532-4010-44d1-a88f-a891cf347058 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:11.825596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa8532-4010-44d1-a88f-a891cf347058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:11.853411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:11.859925 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-dfaa8532-4010-44d1-a88f-a891cf347058 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:11.862024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-dfaa8532-4010-44d1-a88f-a891cf347058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:11.923129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c6590-62a4-43b4-b431-50095119d472 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:11.963516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93678596-97b5-4608-ac37-4af930f18b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:11.964988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93678596-97b5-4608-ac37-4af930f18b3d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:11.965262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93678596-97b5-4608-ac37-4af930f18b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:11.965525 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93678596-97b5-4608-ac37-4af930f18b3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:11.966248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93678596-97b5-4608-ac37-4af930f18b3d None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=0a726949e8cc4cfdb25aa78f2761cff5) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:11.966552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93678596-97b5-4608-ac37-4af930f18b3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:11.967991 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93678596-97b5-4608-ac37-4af930f18b3d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:11.985348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 355/691] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:17:11 2022] GET /identity/v3/role_assignments?user.id=0a726949e8cc4cfdb25aa78f2761cff5&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:11.997008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 337/692] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:17:12.082835 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f875c3-afff-4c6e-a07c-f77966c40ea2 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:12.083126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f875c3-afff-4c6e-a07c-f77966c40ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:12.083556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f875c3-afff-4c6e-a07c-f77966c40ea2 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:12.085138 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 356/693] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:12.091215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c359a-5260-4665-b448-9dfaf59abaa3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:12.091531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c359a-5260-4665-b448-9dfaf59abaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:12.091786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c359a-5260-4665-b448-9dfaf59abaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:12.118120 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-2f5c359a-5260-4665-b448-9dfaf59abaa3 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:12.119338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2f5c359a-5260-4665-b448-9dfaf59abaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:12.159559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:12.218942 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 338/694] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:17:12.306735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f875c3-afff-4c6e-a07c-f77966c40ea2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:12.416537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f875c3-afff-4c6e-a07c-f77966c40ea2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:12.435327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:12.454743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97abf21-1309-4de5-a054-634116180908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:12.455748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97abf21-1309-4de5-a054-634116180908 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:12.456107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97abf21-1309-4de5-a054-634116180908 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:12.456377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97abf21-1309-4de5-a054-634116180908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:12.467274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97abf21-1309-4de5-a054-634116180908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:12.468784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97abf21-1309-4de5-a054-634116180908 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e97abf21-1309-4de5-a054-634116180908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:12.484268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:12.485233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:12.486507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:12.486507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:12.486507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:12.486507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 357/695] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:12.491979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c359a-5260-4665-b448-9dfaf59abaa3 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:12.550971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97abf21-1309-4de5-a054-634116180908 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:12.615791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c359a-5260-4665-b448-9dfaf59abaa3 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:12.663365 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5ab77-7f5f-4438-984a-748b8ff7fb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:12.664708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5ab77-7f5f-4438-984a-748b8ff7fb19 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:12.665203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5ab77-7f5f-4438-984a-748b8ff7fb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:12.665936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5ab77-7f5f-4438-984a-748b8ff7fb19 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:12.666869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5ab77-7f5f-4438-984a-748b8ff7fb19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:12.667485 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5ab77-7f5f-4438-984a-748b8ff7fb19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:12.669626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5ab77-7f5f-4438-984a-748b8ff7fb19 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:12.684548 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 339/696] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:12.690837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5ab77-7f5f-4438-984a-748b8ff7fb19 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:12.696030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97abf21-1309-4de5-a054-634116180908 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:12.720364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:12.760320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81727f-8f63-4e6c-8779-9ccfc6dea33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:12.760964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81727f-8f63-4e6c-8779-9ccfc6dea33f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:12.761243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81727f-8f63-4e6c-8779-9ccfc6dea33f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:12.761516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81727f-8f63-4e6c-8779-9ccfc6dea33f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:12.781840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81727f-8f63-4e6c-8779-9ccfc6dea33f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-6c81727f-8f63-4e6c-8779-9ccfc6dea33f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:12.806666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:12.807603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:12.808484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:12.808484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:12.808484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:12.808484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:12.808484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:12.808484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:12.808484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:12.808484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:12.808484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:12.808484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:12.808484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:17:12.808484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:12.809731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 358/697] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:17:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:12.815039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81727f-8f63-4e6c-8779-9ccfc6dea33f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:12.879499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:12.896283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5ab77-7f5f-4438-984a-748b8ff7fb19 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:12.957794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da82d6-c623-4a01-a5ac-93d29d2ace8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:12.958335 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da82d6-c623-4a01-a5ac-93d29d2ace8e None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:12.958622 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da82d6-c623-4a01-a5ac-93d29d2ace8e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:12.958903 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da82d6-c623-4a01-a5ac-93d29d2ace8e None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:12.959558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da82d6-c623-4a01-a5ac-93d29d2ace8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:12.962069 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da82d6-c623-4a01-a5ac-93d29d2ace8e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:12.995345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 340/698] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:17:12 2022] POST /identity/v3/users => generated 254 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:17:13.009076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81727f-8f63-4e6c-8779-9ccfc6dea33f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:13.047169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af92c4-7722-477a-8292-521c275644d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:13.047868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af92c4-7722-477a-8292-521c275644d7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:13.048189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af92c4-7722-477a-8292-521c275644d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:13.048463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af92c4-7722-477a-8292-521c275644d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:13.049305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af92c4-7722-477a-8292-521c275644d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:13.049616 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af92c4-7722-477a-8292-521c275644d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:13.051021 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af92c4-7722-477a-8292-521c275644d7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:13.061009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 359/699] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:17:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:13.069199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da82d6-c623-4a01-a5ac-93d29d2ace8e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:13.160467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:13.193969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da82d6-c623-4a01-a5ac-93d29d2ace8e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:13.232617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6eab2-060e-445c-aeaf-11d0f8788b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:13.233248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6eab2-060e-445c-aeaf-11d0f8788b22 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:13.233535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6eab2-060e-445c-aeaf-11d0f8788b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:13.233810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6eab2-060e-445c-aeaf-11d0f8788b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:13.245930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6eab2-060e-445c-aeaf-11d0f8788b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:13.247445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6eab2-060e-445c-aeaf-11d0f8788b22 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-73b6eab2-060e-445c-aeaf-11d0f8788b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:13.264886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:13.265920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:13.267208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:13.267208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:13.267208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:13.267208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 341/700] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:13.272166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af92c4-7722-477a-8292-521c275644d7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:13.371311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:13.380614 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af92c4-7722-477a-8292-521c275644d7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:13.421835 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959de93f-a8f4-4c7e-ac57-0e5106995a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:13.422451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959de93f-a8f4-4c7e-ac57-0e5106995a20 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:13.422731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959de93f-a8f4-4c7e-ac57-0e5106995a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:13.423001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959de93f-a8f4-4c7e-ac57-0e5106995a20 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:13.424678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959de93f-a8f4-4c7e-ac57-0e5106995a20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:13.425054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959de93f-a8f4-4c7e-ac57-0e5106995a20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:13.426421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959de93f-a8f4-4c7e-ac57-0e5106995a20 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:13.443449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 360/701] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:13.449377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6eab2-060e-445c-aeaf-11d0f8788b22 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:13.573350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6eab2-060e-445c-aeaf-11d0f8788b22 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:13.616454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:13.631180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d92ff2-d44a-4d19-9d6f-21609fbcb526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:13.631839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d92ff2-d44a-4d19-9d6f-21609fbcb526 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:13.632160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d92ff2-d44a-4d19-9d6f-21609fbcb526 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:13.632440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d92ff2-d44a-4d19-9d6f-21609fbcb526 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:13.646874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d92ff2-d44a-4d19-9d6f-21609fbcb526 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:13.648366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d92ff2-d44a-4d19-9d6f-21609fbcb526 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-77d92ff2-d44a-4d19-9d6f-21609fbcb526 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:13.661399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:13.663655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:13.666019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:13.666019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:13.666019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:13.666019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:13.666019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:13.666019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:13.666019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:13.666019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:13.666019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:17:13.666019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:13.669090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959de93f-a8f4-4c7e-ac57-0e5106995a20 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:13.669243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 342/702] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 05:17:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:13.710899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146ce47-d979-4d9f-bd96-2171ac5f0e29 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:13.711203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146ce47-d979-4d9f-bd96-2171ac5f0e29 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:13.711453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146ce47-d979-4d9f-bd96-2171ac5f0e29 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:13.713199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 343/703] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:13.718337 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617c1b9-29b6-49f5-9698-24401041d3ac None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:13.718624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617c1b9-29b6-49f5-9698-24401041d3ac None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:13.718900 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617c1b9-29b6-49f5-9698-24401041d3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:13.739843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8617c1b9-29b6-49f5-9698-24401041d3ac None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:13.741153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8617c1b9-29b6-49f5-9698-24401041d3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:13.813442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959de93f-a8f4-4c7e-ac57-0e5106995a20 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:13.830093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 344/704] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:17:13.833489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1746c-fad3-4b48-b3b4-ca20e2dbc6b2 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:13.833777 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1746c-fad3-4b48-b3b4-ca20e2dbc6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:13.834052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1746c-fad3-4b48-b3b4-ca20e2dbc6b2 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:13.835584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 345/705] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:13.840940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a5564-a5d5-48e9-a5c7-e77016d9c84a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:13.840940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a5564-a5d5-48e9-a5c7-e77016d9c84a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:13.841316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a5564-a5d5-48e9-a5c7-e77016d9c84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:13.869342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-cc5a5564-a5d5-48e9-a5c7-e77016d9c84a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:13.870533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cc5a5564-a5d5-48e9-a5c7-e77016d9c84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:13.879434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9d645-5f88-4c51-b65d-56fdf333120b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:13.880081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9d645-5f88-4c51-b65d-56fdf333120b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:13.880695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9d645-5f88-4c51-b65d-56fdf333120b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:13.880695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9d645-5f88-4c51-b65d-56fdf333120b None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:13.881334 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9d645-5f88-4c51-b65d-56fdf333120b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:13.881638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9d645-5f88-4c51-b65d-56fdf333120b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:13.883056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9d645-5f88-4c51-b65d-56fdf333120b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:13.915031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 361/706] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 05:17:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:13.917540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9d645-5f88-4c51-b65d-56fdf333120b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:13.933317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:14.030734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 346/707] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 05:17:14.053190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9d645-5f88-4c51-b65d-56fdf333120b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:14.088218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e505fb2-03d2-491c-a513-1ba13a723c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:14.089123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e505fb2-03d2-491c-a513-1ba13a723c34 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:14.089612 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e505fb2-03d2-491c-a513-1ba13a723c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:14.090162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e505fb2-03d2-491c-a513-1ba13a723c34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:14.159011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e505fb2-03d2-491c-a513-1ba13a723c34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:14.160854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e505fb2-03d2-491c-a513-1ba13a723c34 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:14.162305 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a5564-a5d5-48e9-a5c7-e77016d9c84a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:14.176671 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:14.214813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:14.278046 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 362/708] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:17:14.283694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e505fb2-03d2-491c-a513-1ba13a723c34 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:14.330788 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a5564-a5d5-48e9-a5c7-e77016d9c84a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:14.404720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe052c8-2101-4df5-b53b-2720917aafc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:14.405775 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe052c8-2101-4df5-b53b-2720917aafc5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:14.406613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe052c8-2101-4df5-b53b-2720917aafc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:14.407221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe052c8-2101-4df5-b53b-2720917aafc5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:14.408180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe052c8-2101-4df5-b53b-2720917aafc5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:14.410255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe052c8-2101-4df5-b53b-2720917aafc5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-1fe052c8-2101-4df5-b53b-2720917aafc5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:14.425187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:14.426268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 05:17:14.427137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:14.427891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e505fb2-03d2-491c-a513-1ba13a723c34 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:14.430650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 347/709] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:14 2022] GET /identity/v3/services/network => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:14.436166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe052c8-2101-4df5-b53b-2720917aafc5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:14.441318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:14.468272 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a42cc-4461-4687-b5b9-db85bfacb525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:14.469001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a42cc-4461-4687-b5b9-db85bfacb525 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:14.469373 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a42cc-4461-4687-b5b9-db85bfacb525 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:14.469737 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a42cc-4461-4687-b5b9-db85bfacb525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:14.483224 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a42cc-4461-4687-b5b9-db85bfacb525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:14.485565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a42cc-4461-4687-b5b9-db85bfacb525 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-a88a42cc-4461-4687-b5b9-db85bfacb525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:14.503901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:14.505185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:14.506300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:14.506300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:14.506300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:14.512038 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 363/710] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:14.512038 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a42cc-4461-4687-b5b9-db85bfacb525 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:14.582185 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe052c8-2101-4df5-b53b-2720917aafc5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:14.594959 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:14.626909 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a42cc-4461-4687-b5b9-db85bfacb525 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:14.627573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40b2ff-49e8-4427-ab16-3d9ccb12d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:14.628870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40b2ff-49e8-4427-ab16-3d9ccb12d888 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:14.630121 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40b2ff-49e8-4427-ab16-3d9ccb12d888 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:14.630606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40b2ff-49e8-4427-ab16-3d9ccb12d888 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:14.631384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40b2ff-49e8-4427-ab16-3d9ccb12d888 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:14.633023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40b2ff-49e8-4427-ab16-3d9ccb12d888 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-8f40b2ff-49e8-4427-ab16-3d9ccb12d888 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:14.645573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:14.646581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 05:17:14.647470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:14.650603 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 348/711] 10.222.0.33 () {56 vars in 1129 bytes} [Sun Sep 11 05:17:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:14.651707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40b2ff-49e8-4427-ab16-3d9ccb12d888 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:14.668691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925a43f-b5e0-4317-93f8-1f1d9cacfa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:14.669512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925a43f-b5e0-4317-93f8-1f1d9cacfa2c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:14.670038 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925a43f-b5e0-4317-93f8-1f1d9cacfa2c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:14.670454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925a43f-b5e0-4317-93f8-1f1d9cacfa2c None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:14.671321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925a43f-b5e0-4317-93f8-1f1d9cacfa2c None admin] RBAC: Adding query filter params (name=network) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:14.671850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925a43f-b5e0-4317-93f8-1f1d9cacfa2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:14.673380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925a43f-b5e0-4317-93f8-1f1d9cacfa2c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:14.684732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 364/712] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 05:17:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:14.685236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925a43f-b5e0-4317-93f8-1f1d9cacfa2c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:14.716541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:14.792043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40b2ff-49e8-4427-ab16-3d9ccb12d888 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:14.802627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925a43f-b5e0-4317-93f8-1f1d9cacfa2c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:14.847252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0788c-328d-418c-ab26-6e97f2dfad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:14.848580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0788c-328d-418c-ab26-6e97f2dfad6c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:14.848822 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad7920-c526-4a5b-84c6-4f38fa77de3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:14.849211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0788c-328d-418c-ab26-6e97f2dfad6c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:14.849607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad7920-c526-4a5b-84c6-4f38fa77de3c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:14.849773 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0788c-328d-418c-ab26-6e97f2dfad6c None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:14.850060 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad7920-c526-4a5b-84c6-4f38fa77de3c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:14.850421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad7920-c526-4a5b-84c6-4f38fa77de3c None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:14.850884 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0788c-328d-418c-ab26-6e97f2dfad6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:14.851232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad7920-c526-4a5b-84c6-4f38fa77de3c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:14.851484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0788c-328d-418c-ab26-6e97f2dfad6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:14.851702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad7920-c526-4a5b-84c6-4f38fa77de3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:14.853191 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad7920-c526-4a5b-84c6-4f38fa77de3c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:14.853446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0788c-328d-418c-ab26-6e97f2dfad6c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:14.858928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 365/713] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 11 05:17:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:14.859911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad7920-c526-4a5b-84c6-4f38fa77de3c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:14.874402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 349/714] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:14.874402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0788c-328d-418c-ab26-6e97f2dfad6c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:14.911958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:14.975702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:14.986742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad7920-c526-4a5b-84c6-4f38fa77de3c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:14.988614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0788c-328d-418c-ab26-6e97f2dfad6c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:15.021940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0ce06-b7e2-453f-b501-3d302d6b4423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:15.022548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0ce06-b7e2-453f-b501-3d302d6b4423 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:15.022823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0ce06-b7e2-453f-b501-3d302d6b4423 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:15.023095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0ce06-b7e2-453f-b501-3d302d6b4423 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:15.023720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0ce06-b7e2-453f-b501-3d302d6b4423 None admin] RBAC: Adding query filter params (type=network) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:15.024048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0ce06-b7e2-453f-b501-3d302d6b4423 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:15.025191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f6e44-8087-4a04-a9f8-c64bee6c2180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:15.025409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0ce06-b7e2-453f-b501-3d302d6b4423 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:15.025790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f6e44-8087-4a04-a9f8-c64bee6c2180 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:15.026066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f6e44-8087-4a04-a9f8-c64bee6c2180 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:15.026336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f6e44-8087-4a04-a9f8-c64bee6c2180 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:15.027014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f6e44-8087-4a04-a9f8-c64bee6c2180 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:15.027319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f6e44-8087-4a04-a9f8-c64bee6c2180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:15.028705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f6e44-8087-4a04-a9f8-c64bee6c2180 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:15.036248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 366/715] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 05:17:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:15.036248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0ce06-b7e2-453f-b501-3d302d6b4423 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:15.054500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 350/716] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 11 05:17:14 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:15.056034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6155e9-d0d6-401a-9cc2-ddad7ff2a9b9 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:15.056323 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6155e9-d0d6-401a-9cc2-ddad7ff2a9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:15.056597 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6155e9-d0d6-401a-9cc2-ddad7ff2a9b9 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:15.062138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 351/717] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:15.071818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfff08b-a640-4cec-a29c-06a6b8f89a7f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:15.072738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfff08b-a640-4cec-a29c-06a6b8f89a7f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:15.073205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfff08b-a640-4cec-a29c-06a6b8f89a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:15.096366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-2dfff08b-a640-4cec-a29c-06a6b8f89a7f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:15.097909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2dfff08b-a640-4cec-a29c-06a6b8f89a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:15.227805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0ce06-b7e2-453f-b501-3d302d6b4423 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:15.273197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b1847-d445-430d-953d-75a3c5a33b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:15.273854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b1847-d445-430d-953d-75a3c5a33b05 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:15.274132 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b1847-d445-430d-953d-75a3c5a33b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:15.274401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b1847-d445-430d-953d-75a3c5a33b05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:15.278294 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 352/718] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:17:15.286454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b1847-d445-430d-953d-75a3c5a33b05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:15.287915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b1847-d445-430d-953d-75a3c5a33b05 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-057b1847-d445-430d-953d-75a3c5a33b05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:15.304197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:15.305131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:15.306016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:15.306016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:15.306016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:15.306016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:15.306016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:15.306016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:15.306016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:15.306016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:15.306016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 367/719] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:15.312595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfff08b-a640-4cec-a29c-06a6b8f89a7f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:15.356296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:15.459437 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfff08b-a640-4cec-a29c-06a6b8f89a7f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:15.506305 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c7cd7-3bfc-4df9-a7e1-adc1045091a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:15.506838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c7cd7-3bfc-4df9-a7e1-adc1045091a1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:15.507119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c7cd7-3bfc-4df9-a7e1-adc1045091a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:15.507391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c7cd7-3bfc-4df9-a7e1-adc1045091a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:15.508114 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c7cd7-3bfc-4df9-a7e1-adc1045091a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:15.508420 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c7cd7-3bfc-4df9-a7e1-adc1045091a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:15.509849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c7cd7-3bfc-4df9-a7e1-adc1045091a1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:15.527171 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 353/720] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:15.533688 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b1847-d445-430d-953d-75a3c5a33b05 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:15.565612 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:15.613522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c7cd7-3bfc-4df9-a7e1-adc1045091a1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:15.685074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b1847-d445-430d-953d-75a3c5a33b05 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:15.724220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870e934-3e9a-43b3-9fde-019f1a0a7e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:15.724833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870e934-3e9a-43b3-9fde-019f1a0a7e1f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:15.725114 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870e934-3e9a-43b3-9fde-019f1a0a7e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:15.725392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870e934-3e9a-43b3-9fde-019f1a0a7e1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:15.726120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870e934-3e9a-43b3-9fde-019f1a0a7e1f None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=3bd5a9a09f034c05815c631a44d1cdd9) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:15.726428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870e934-3e9a-43b3-9fde-019f1a0a7e1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:15.727884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870e934-3e9a-43b3-9fde-019f1a0a7e1f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:15.734713 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 368/721] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:17:15 2022] GET /identity/v3/role_assignments?user.id=3bd5a9a09f034c05815c631a44d1cdd9&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:15.784569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:15.810421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c7cd7-3bfc-4df9-a7e1-adc1045091a1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:15.879148 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bfc08-61c4-4a6c-8215-d8a4b6833376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:15.879668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0964cc-3565-40d3-9f9b-ae8739f70817 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:15.879781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bfc08-61c4-4a6c-8215-d8a4b6833376 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:15.880097 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bfc08-61c4-4a6c-8215-d8a4b6833376 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:15.880375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bfc08-61c4-4a6c-8215-d8a4b6833376 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:15.880700 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0964cc-3565-40d3-9f9b-ae8739f70817 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:15.881163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0964cc-3565-40d3-9f9b-ae8739f70817 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:15.883041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 369/722] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:15.888928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36d8c5-9a5a-4ddb-829b-bfbfe27f20ed None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:15.889539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36d8c5-9a5a-4ddb-829b-bfbfe27f20ed None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:15.890012 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36d8c5-9a5a-4ddb-829b-bfbfe27f20ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:15.903310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bfc08-61c4-4a6c-8215-d8a4b6833376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-134bfc08-61c4-4a6c-8215-d8a4b6833376 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:15.908928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:15.909965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:15.910830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:15.910830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:15.910830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:15.910830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:15.910830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:15.910830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:15.910830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:15.910830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:15.910830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:15.910830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:15.910830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:17:15.910830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:15.910830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 354/723] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:17:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:15.917896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bfc08-61c4-4a6c-8215-d8a4b6833376 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:15.920512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-cb36d8c5-9a5a-4ddb-829b-bfbfe27f20ed None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:15.922058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cb36d8c5-9a5a-4ddb-829b-bfbfe27f20ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:16.055839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bfc08-61c4-4a6c-8215-d8a4b6833376 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:16.063897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 370/724] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:17:16.092514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deaded-6432-4597-ba11-cc75329c9aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:16.093186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deaded-6432-4597-ba11-cc75329c9aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:16.093487 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deaded-6432-4597-ba11-cc75329c9aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:16.093850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deaded-6432-4597-ba11-cc75329c9aa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:16.094569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03deaded-6432-4597-ba11-cc75329c9aa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:16.094933 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03deaded-6432-4597-ba11-cc75329c9aa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:16.096982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03deaded-6432-4597-ba11-cc75329c9aa4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:16.106457 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 355/725] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:17:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:16.112435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36d8c5-9a5a-4ddb-829b-bfbfe27f20ed None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:16.134430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:16.266409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36d8c5-9a5a-4ddb-829b-bfbfe27f20ed None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:16.316056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff4fc4-c599-47d0-864f-ba7280d325b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:16.316702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff4fc4-c599-47d0-864f-ba7280d325b1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:16.316981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff4fc4-c599-47d0-864f-ba7280d325b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:16.317251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff4fc4-c599-47d0-864f-ba7280d325b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:16.330170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff4fc4-c599-47d0-864f-ba7280d325b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:16.331831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff4fc4-c599-47d0-864f-ba7280d325b1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d2ff4fc4-c599-47d0-864f-ba7280d325b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:16.347188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:16.348282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:16.349218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:16.349218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:16.349218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:16.349218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 371/726] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:16.357091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deaded-6432-4597-ba11-cc75329c9aa4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:16.375614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:16.491872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deaded-6432-4597-ba11-cc75329c9aa4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:16.537082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669fe81-edb7-4fbf-a4c4-e25556acad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:16.538048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669fe81-edb7-4fbf-a4c4-e25556acad69 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:16.538548 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669fe81-edb7-4fbf-a4c4-e25556acad69 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:16.539041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669fe81-edb7-4fbf-a4c4-e25556acad69 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:16.539875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669fe81-edb7-4fbf-a4c4-e25556acad69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:16.540404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669fe81-edb7-4fbf-a4c4-e25556acad69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:16.541998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669fe81-edb7-4fbf-a4c4-e25556acad69 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:16.542110 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff4fc4-c599-47d0-864f-ba7280d325b1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:16.561340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 356/727] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:16.570542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669fe81-edb7-4fbf-a4c4-e25556acad69 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:16.664502 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:16.725666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:16.742293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff4fc4-c599-47d0-864f-ba7280d325b1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:16.771112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669fe81-edb7-4fbf-a4c4-e25556acad69 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:16.815553 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53d94c-c7e2-47bd-b1e7-ab9a993fa004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:16.816492 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53d94c-c7e2-47bd-b1e7-ab9a993fa004 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:16.816973 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53d94c-c7e2-47bd-b1e7-ab9a993fa004 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:16.817444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53d94c-c7e2-47bd-b1e7-ab9a993fa004 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:16.826197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53d94c-c7e2-47bd-b1e7-ab9a993fa004 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:16.827871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53d94c-c7e2-47bd-b1e7-ab9a993fa004 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ea53d94c-c7e2-47bd-b1e7-ab9a993fa004 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:16.844662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:16.846458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:16.847578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:16.847578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:16.847578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:16.847578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:16.847578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:16.847578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:16.847578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:16.847578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:16.847578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 05:17:16.847578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:16.850236 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 357/728] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:17:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:16.855900 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53d94c-c7e2-47bd-b1e7-ab9a993fa004 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:16.856679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80c6e0-456f-40e0-8dbd-e2055ada047f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:16.857326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80c6e0-456f-40e0-8dbd-e2055ada047f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:16.857611 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80c6e0-456f-40e0-8dbd-e2055ada047f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:16.857883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80c6e0-456f-40e0-8dbd-e2055ada047f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:16.892938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80c6e0-456f-40e0-8dbd-e2055ada047f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-bb80c6e0-456f-40e0-8dbd-e2055ada047f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:16.898578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:16.899694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:16.900776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:16.900776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:16.900776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:16.900776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:16.900776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:16.900776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:16.900776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:16.900776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:16.900776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:16.900776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:16.900776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:16.900776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:16.908594 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 372/729] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:17:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:16.909458 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80c6e0-456f-40e0-8dbd-e2055ada047f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:16.979784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53d94c-c7e2-47bd-b1e7-ab9a993fa004 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:17.016724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:17.026289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe1002-4a41-4e9e-aab5-a633b90a6334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:17.027093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe1002-4a41-4e9e-aab5-a633b90a6334 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:17.027518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe1002-4a41-4e9e-aab5-a633b90a6334 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:17.027932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe1002-4a41-4e9e-aab5-a633b90a6334 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:17.029370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe1002-4a41-4e9e-aab5-a633b90a6334 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:17.029814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe1002-4a41-4e9e-aab5-a633b90a6334 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:17.031378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe1002-4a41-4e9e-aab5-a633b90a6334 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:17.055093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 358/730] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 11 05:17:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:17.061319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe1002-4a41-4e9e-aab5-a633b90a6334 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:17.174850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80c6e0-456f-40e0-8dbd-e2055ada047f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:17.191621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe1002-4a41-4e9e-aab5-a633b90a6334 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:17.232534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:17.234032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64ade4-443a-471c-8a5b-5cfa6ac969fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:17.234817 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64ade4-443a-471c-8a5b-5cfa6ac969fe None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:17.235330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64ade4-443a-471c-8a5b-5cfa6ac969fe None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:17.235746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64ade4-443a-471c-8a5b-5cfa6ac969fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:17.268727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809705e-c0b0-403f-bcc6-5f13d15d6eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:17.269396 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809705e-c0b0-403f-bcc6-5f13d15d6eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:17.269630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809705e-c0b0-403f-bcc6-5f13d15d6eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:17.269915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809705e-c0b0-403f-bcc6-5f13d15d6eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:17.270576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809705e-c0b0-403f-bcc6-5f13d15d6eb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:17.271051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809705e-c0b0-403f-bcc6-5f13d15d6eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:17.272286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809705e-c0b0-403f-bcc6-5f13d15d6eb2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:17.289177 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 373/731] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:17:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:17.294194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809705e-c0b0-403f-bcc6-5f13d15d6eb2 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:17.303889 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:17.319821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64ade4-443a-471c-8a5b-5cfa6ac969fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:17.321607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64ade4-443a-471c-8a5b-5cfa6ac969fe None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:17.425094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809705e-c0b0-403f-bcc6-5f13d15d6eb2 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:17.458400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553281d-7a3a-4bb3-bc12-3132b8b047b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:17.459071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553281d-7a3a-4bb3-bc12-3132b8b047b0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:17.459295 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553281d-7a3a-4bb3-bc12-3132b8b047b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:17.459565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553281d-7a3a-4bb3-bc12-3132b8b047b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:17.470792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553281d-7a3a-4bb3-bc12-3132b8b047b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:17.472431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553281d-7a3a-4bb3-bc12-3132b8b047b0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-1553281d-7a3a-4bb3-bc12-3132b8b047b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:17.488151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:17.489321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:17.490291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:17.490291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:17.490291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:17.490291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 374/732] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:17.497338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553281d-7a3a-4bb3-bc12-3132b8b047b0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:17.499107 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 359/733] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 05:17:17.506295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64ade4-443a-471c-8a5b-5cfa6ac969fe None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:17.568392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:17.676814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553281d-7a3a-4bb3-bc12-3132b8b047b0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:17.678842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:17.703323 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64ade4-443a-471c-8a5b-5cfa6ac969fe None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:17.735246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d390f6-bf9a-47b9-9a0d-942bcfafa012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:17.736107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d390f6-bf9a-47b9-9a0d-942bcfafa012 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:17.736483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d390f6-bf9a-47b9-9a0d-942bcfafa012 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:17.737114 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d390f6-bf9a-47b9-9a0d-942bcfafa012 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:17.737940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d390f6-bf9a-47b9-9a0d-942bcfafa012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:17.738319 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d390f6-bf9a-47b9-9a0d-942bcfafa012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:17.739817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d390f6-bf9a-47b9-9a0d-942bcfafa012 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:17.749027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24e7e2-8a2a-4f5c-931a-a9e1f13f08b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:17.749947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24e7e2-8a2a-4f5c-931a-a9e1f13f08b8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:17.750440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24e7e2-8a2a-4f5c-931a-a9e1f13f08b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:17.750898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24e7e2-8a2a-4f5c-931a-a9e1f13f08b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:17.753830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 375/734] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:17.755358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12515d9-e24e-425f-aef6-a272dedbf969 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:17.755709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12515d9-e24e-425f-aef6-a272dedbf969 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:17.756083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12515d9-e24e-425f-aef6-a272dedbf969 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:17.760081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 376/735] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:17.761045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12515d9-e24e-425f-aef6-a272dedbf969 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:17.764948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24e7e2-8a2a-4f5c-931a-a9e1f13f08b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:17.766685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24e7e2-8a2a-4f5c-931a-a9e1f13f08b8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-fe24e7e2-8a2a-4f5c-931a-a9e1f13f08b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:17.786643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:17.787681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:17.788656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:17.788656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:17.788656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:17.788656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 360/736] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:17.792198 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83a21e-0617-4b4f-b987-ac99a72e6d62 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:17.792198 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83a21e-0617-4b4f-b987-ac99a72e6d62 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:17.792198 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83a21e-0617-4b4f-b987-ac99a72e6d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:17.819423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-af83a21e-0617-4b4f-b987-ac99a72e6d62 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:17.821553 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-af83a21e-0617-4b4f-b987-ac99a72e6d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:17.886701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:17.899540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12515d9-e24e-425f-aef6-a272dedbf969 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:17.911607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 361/737] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:17:17.912801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83a21e-0617-4b4f-b987-ac99a72e6d62 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:17.946606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ffb2d-f86a-4c71-aaae-04c5af699cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:17.947322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ffb2d-f86a-4c71-aaae-04c5af699cea None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:17.947650 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ffb2d-f86a-4c71-aaae-04c5af699cea None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:17.947986 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ffb2d-f86a-4c71-aaae-04c5af699cea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:17.956665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ffb2d-f86a-4c71-aaae-04c5af699cea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:17.958298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ffb2d-f86a-4c71-aaae-04c5af699cea None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-9b4ffb2d-f86a-4c71-aaae-04c5af699cea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:17.970728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:17.971720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:17.972724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:17.972724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:17.972724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:17.972724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:17.972724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:17.972724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:17.972724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:17.972724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:17.972724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 05:17:17.972724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:17.972724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 377/738] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 05:17:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:17.973954 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c59e0-6660-48f8-9809-bbeb23aeb5b0 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:17.974332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c59e0-6660-48f8-9809-bbeb23aeb5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:17.974706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c59e0-6660-48f8-9809-bbeb23aeb5b0 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:17.976554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 378/739] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:17.977342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c59e0-6660-48f8-9809-bbeb23aeb5b0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:18.014844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:18.064363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83a21e-0617-4b4f-b987-ac99a72e6d62 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:18.099839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46735174-c839-4538-a8ec-1e7506e5e812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:18.100667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735174-c839-4538-a8ec-1e7506e5e812 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:18.101035 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735174-c839-4538-a8ec-1e7506e5e812 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:18.101407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735174-c839-4538-a8ec-1e7506e5e812 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:18.102208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46735174-c839-4538-a8ec-1e7506e5e812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:18.102607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46735174-c839-4538-a8ec-1e7506e5e812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:18.104096 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46735174-c839-4538-a8ec-1e7506e5e812 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:18.107415 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c59e0-6660-48f8-9809-bbeb23aeb5b0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:18.121524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 362/740] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:18.121524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912b82c-4ec5-485d-8b25-96b2eddff21f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:18.121524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912b82c-4ec5-485d-8b25-96b2eddff21f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:18.121524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912b82c-4ec5-485d-8b25-96b2eddff21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:18.138753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4912b82c-4ec5-485d-8b25-96b2eddff21f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:18.139719 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4912b82c-4ec5-485d-8b25-96b2eddff21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:18.156101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e1b09-5185-4f3f-a46a-d1dbd36568d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:18.156715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e1b09-5185-4f3f-a46a-d1dbd36568d0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:18.157120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e1b09-5185-4f3f-a46a-d1dbd36568d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:18.157248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e1b09-5185-4f3f-a46a-d1dbd36568d0 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:18.158180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e1b09-5185-4f3f-a46a-d1dbd36568d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:18.158492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e1b09-5185-4f3f-a46a-d1dbd36568d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:18.159914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e1b09-5185-4f3f-a46a-d1dbd36568d0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:18.178443 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 379/741] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 11 05:17:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:18.179150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e1b09-5185-4f3f-a46a-d1dbd36568d0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:18.193962 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:18.215086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 363/742] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:17:18.216304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80395728-68ca-45bd-b491-47bb992c299d None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:18.216588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80395728-68ca-45bd-b491-47bb992c299d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:18.216851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80395728-68ca-45bd-b491-47bb992c299d None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:18.218234 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 364/743] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:18.221361 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80395728-68ca-45bd-b491-47bb992c299d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:18.284073 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:18.285497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e1b09-5185-4f3f-a46a-d1dbd36568d0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:18.320731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c3487-2fc3-4a35-9f8f-88907f39ff29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:18.321369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c3487-2fc3-4a35-9f8f-88907f39ff29 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:18.321646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c3487-2fc3-4a35-9f8f-88907f39ff29 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:18.321915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c3487-2fc3-4a35-9f8f-88907f39ff29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:18.327798 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80395728-68ca-45bd-b491-47bb992c299d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:18.329268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c3487-2fc3-4a35-9f8f-88907f39ff29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:18.331248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c3487-2fc3-4a35-9f8f-88907f39ff29 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-2d7c3487-2fc3-4a35-9f8f-88907f39ff29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:18.342675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:18.343577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:18.344499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:18.344499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:18.344499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:18.344499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:18.344499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:18.344499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:18.344499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:18.344499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:18.349011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 380/744] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:18.349660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c3487-2fc3-4a35-9f8f-88907f39ff29 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:18.369066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0183aed-3e70-4e7e-92f4-57501dcb0aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:18.369676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0183aed-3e70-4e7e-92f4-57501dcb0aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:18.369957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0183aed-3e70-4e7e-92f4-57501dcb0aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:18.370230 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0183aed-3e70-4e7e-92f4-57501dcb0aa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:18.423060 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:18.444965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0183aed-3e70-4e7e-92f4-57501dcb0aa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:18.446482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0183aed-3e70-4e7e-92f4-57501dcb0aa5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:18.453270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c3487-2fc3-4a35-9f8f-88907f39ff29 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:18.488817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089734c-e6e3-4e8a-996e-0be1e6ec8d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:18.489357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089734c-e6e3-4e8a-996e-0be1e6ec8d89 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:18.489634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089734c-e6e3-4e8a-996e-0be1e6ec8d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:18.489906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089734c-e6e3-4e8a-996e-0be1e6ec8d89 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:18.490556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089734c-e6e3-4e8a-996e-0be1e6ec8d89 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:18.492102 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089734c-e6e3-4e8a-996e-0be1e6ec8d89 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:18.508492 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:18.512059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 381/745] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:17:18 2022] POST /identity/v3/services => generated 251 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:17:18.512059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0afe3-3bb0-4a2b-b231-c02778f5f782 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:18.512059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0afe3-3bb0-4a2b-b231-c02778f5f782 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:18.512059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0afe3-3bb0-4a2b-b231-c02778f5f782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:18.535564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-16c0afe3-3bb0-4a2b-b231-c02778f5f782 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:18.537226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-16c0afe3-3bb0-4a2b-b231-c02778f5f782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:18.562307 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 365/746] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 05:17:18.563310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0183aed-3e70-4e7e-92f4-57501dcb0aa5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:18.635060 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 382/747] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 05:17:18.636028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0afe3-3bb0-4a2b-b231-c02778f5f782 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:18.683243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0183aed-3e70-4e7e-92f4-57501dcb0aa5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:18.714001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:18.729328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25436a56-54f6-4c91-86db-f9ae4e4e13e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:18.729328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25436a56-54f6-4c91-86db-f9ae4e4e13e9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:18.729328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25436a56-54f6-4c91-86db-f9ae4e4e13e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:18.729664 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25436a56-54f6-4c91-86db-f9ae4e4e13e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:18.730396 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25436a56-54f6-4c91-86db-f9ae4e4e13e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:18.730706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25436a56-54f6-4c91-86db-f9ae4e4e13e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:18.732966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25436a56-54f6-4c91-86db-f9ae4e4e13e9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:18.746664 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 366/748] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:18.747423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25436a56-54f6-4c91-86db-f9ae4e4e13e9 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:18.773974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0afe3-3bb0-4a2b-b231-c02778f5f782 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:18.775274 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:18.819948 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb802eac-7de9-44a9-8c41-44ff45ee93fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:18.820588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb802eac-7de9-44a9-8c41-44ff45ee93fd None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:18.820866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb802eac-7de9-44a9-8c41-44ff45ee93fd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:18.821664 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb802eac-7de9-44a9-8c41-44ff45ee93fd None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:18.822321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb802eac-7de9-44a9-8c41-44ff45ee93fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:18.824156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb802eac-7de9-44a9-8c41-44ff45ee93fd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:18.843536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 383/749] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:17:18 2022] POST /identity/v3/services => generated 240 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:17:18.844474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb802eac-7de9-44a9-8c41-44ff45ee93fd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:18.882702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25436a56-54f6-4c91-86db-f9ae4e4e13e9 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:18.938855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e81a8f-79a5-4467-b03e-42a1689602dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:18.940282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e81a8f-79a5-4467-b03e-42a1689602dd None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:18.940571 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e81a8f-79a5-4467-b03e-42a1689602dd None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:18.940844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e81a8f-79a5-4467-b03e-42a1689602dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:18.953811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e81a8f-79a5-4467-b03e-42a1689602dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:18.955955 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e81a8f-79a5-4467-b03e-42a1689602dd None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c5e81a8f-79a5-4467-b03e-42a1689602dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:18.973800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:18.975041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:18.976193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:18.976193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:18.976193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:18.976193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 367/750] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:18.976444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e81a8f-79a5-4467-b03e-42a1689602dd None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:18.982532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb802eac-7de9-44a9-8c41-44ff45ee93fd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:19.004123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:19.025557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebab24f-09b1-4a35-ba90-c472ced2978c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:19.026237 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebab24f-09b1-4a35-ba90-c472ced2978c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:19.026515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebab24f-09b1-4a35-ba90-c472ced2978c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:19.026826 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebab24f-09b1-4a35-ba90-c472ced2978c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:19.027578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebab24f-09b1-4a35-ba90-c472ced2978c None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=0a726949e8cc4cfdb25aa78f2761cff5) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:19.027885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebab24f-09b1-4a35-ba90-c472ced2978c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:19.029608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebab24f-09b1-4a35-ba90-c472ced2978c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:19.038432 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 384/751] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:17:18 2022] GET /identity/v3/role_assignments?user.id=0a726949e8cc4cfdb25aa78f2761cff5&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:19.039314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebab24f-09b1-4a35-ba90-c472ced2978c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:19.100327 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:19.154715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e81a8f-79a5-4467-b03e-42a1689602dd None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:19.182408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebab24f-09b1-4a35-ba90-c472ced2978c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:19.203768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096cfd1-fb03-435e-8b78-41bd20ca1f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:19.204463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096cfd1-fb03-435e-8b78-41bd20ca1f97 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:19.204822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096cfd1-fb03-435e-8b78-41bd20ca1f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:19.205100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096cfd1-fb03-435e-8b78-41bd20ca1f97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:19.217828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096cfd1-fb03-435e-8b78-41bd20ca1f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-5096cfd1-fb03-435e-8b78-41bd20ca1f97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:19.225252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:19.226377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:19.227348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:19.227348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:19.227348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:19.227348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:19.227348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:19.227348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:19.227348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:19.227348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:19.227348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:19.227348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:19.227348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:17:19.227348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:19.228219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 368/752] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:17:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:19.230763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7732e-7d25-44b4-93cf-67b736893c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:19.231458 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7732e-7d25-44b4-93cf-67b736893c8a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:19.231813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7732e-7d25-44b4-93cf-67b736893c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:19.232202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7732e-7d25-44b4-93cf-67b736893c8a None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:19.232860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7732e-7d25-44b4-93cf-67b736893c8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:19.233245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7732e-7d25-44b4-93cf-67b736893c8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:19.234741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7732e-7d25-44b4-93cf-67b736893c8a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:19.235523 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096cfd1-fb03-435e-8b78-41bd20ca1f97 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:19.251152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 385/753] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:19.257955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7732e-7d25-44b4-93cf-67b736893c8a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:19.315537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:19.361531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096cfd1-fb03-435e-8b78-41bd20ca1f97 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:19.375324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7732e-7d25-44b4-93cf-67b736893c8a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:19.393868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:19.403526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a70abe-3cd2-4054-9f14-6f7e5d9a00d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:19.404421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a70abe-3cd2-4054-9f14-6f7e5d9a00d4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:19.404984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a70abe-3cd2-4054-9f14-6f7e5d9a00d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:19.405413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a70abe-3cd2-4054-9f14-6f7e5d9a00d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:19.406347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a70abe-3cd2-4054-9f14-6f7e5d9a00d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:19.406800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a70abe-3cd2-4054-9f14-6f7e5d9a00d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:19.409048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a70abe-3cd2-4054-9f14-6f7e5d9a00d4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:19.414164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac332f-47d7-4027-ad23-f33eefccca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:19.414806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac332f-47d7-4027-ad23-f33eefccca27 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:19.415092 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac332f-47d7-4027-ad23-f33eefccca27 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:19.415695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac332f-47d7-4027-ad23-f33eefccca27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:19.419481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 369/754] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:17:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:19.424088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac332f-47d7-4027-ad23-f33eefccca27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:19.424088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac332f-47d7-4027-ad23-f33eefccca27 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:19.425370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a70abe-3cd2-4054-9f14-6f7e5d9a00d4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-c1ac332f-47d7-4027-ad23-f33eefccca27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:19.435496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:19.436644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:19.437590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:19.437590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:19.437590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:19.437590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:19.437590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:19.437590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:19.437590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:19.437590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:19.437590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 386/755] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:19.442921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac332f-47d7-4027-ad23-f33eefccca27 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:19.622336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac332f-47d7-4027-ad23-f33eefccca27 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:19.624211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:19.647783 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a70abe-3cd2-4054-9f14-6f7e5d9a00d4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:19.664840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f679a12-3657-491d-8799-247851500193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:19.665460 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f679a12-3657-491d-8799-247851500193 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:19.665762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f679a12-3657-491d-8799-247851500193 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:19.666482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f679a12-3657-491d-8799-247851500193 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:19.667189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f679a12-3657-491d-8799-247851500193 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:19.667497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f679a12-3657-491d-8799-247851500193 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:19.668950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f679a12-3657-491d-8799-247851500193 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:19.682428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 387/756] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:19.687963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f679a12-3657-491d-8799-247851500193 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:19.728143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9d59d-f28d-46e1-aa61-3617c5031d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:19.728143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9d59d-f28d-46e1-aa61-3617c5031d6c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:19.728143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9d59d-f28d-46e1-aa61-3617c5031d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:19.728143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9d59d-f28d-46e1-aa61-3617c5031d6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:19.740029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9d59d-f28d-46e1-aa61-3617c5031d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:19.741805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9d59d-f28d-46e1-aa61-3617c5031d6c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-4ea9d59d-f28d-46e1-aa61-3617c5031d6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:19.759213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:19.760227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:19.761174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:19.761174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:19.761174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:19.761174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 370/757] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:19.766221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9d59d-f28d-46e1-aa61-3617c5031d6c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:19.830410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:19.895850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9d59d-f28d-46e1-aa61-3617c5031d6c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:19.934490 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254f17a-e27f-4a09-9d98-dd354d86ceef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:19.935349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254f17a-e27f-4a09-9d98-dd354d86ceef None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:19.935778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254f17a-e27f-4a09-9d98-dd354d86ceef None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:19.936230 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254f17a-e27f-4a09-9d98-dd354d86ceef None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:19.937646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3254f17a-e27f-4a09-9d98-dd354d86ceef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:19.938092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3254f17a-e27f-4a09-9d98-dd354d86ceef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:19.940201 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3254f17a-e27f-4a09-9d98-dd354d86ceef None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:19.952830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f679a12-3657-491d-8799-247851500193 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:19.957823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 371/758] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:19.963513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254f17a-e27f-4a09-9d98-dd354d86ceef None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:20.014294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271e285-2882-4f51-ab57-d030f5bfdcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:20.014910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271e285-2882-4f51-ab57-d030f5bfdcc9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:20.015192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271e285-2882-4f51-ab57-d030f5bfdcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:20.015465 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271e285-2882-4f51-ab57-d030f5bfdcc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:20.016216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271e285-2882-4f51-ab57-d030f5bfdcc9 None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=afd0a6d3b2194604a46767cfde12097b) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:20.016528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271e285-2882-4f51-ab57-d030f5bfdcc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:20.018056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271e285-2882-4f51-ab57-d030f5bfdcc9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:20.028996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 388/759] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:17:19 2022] GET /identity/v3/role_assignments?user.id=afd0a6d3b2194604a46767cfde12097b&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 259 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:20.125520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254f17a-e27f-4a09-9d98-dd354d86ceef None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:20.127669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:20.182337 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f6873-4156-4a2a-9d9b-389f0e0d9d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:20.183241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f6873-4156-4a2a-9d9b-389f0e0d9d64 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:20.183670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f6873-4156-4a2a-9d9b-389f0e0d9d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:20.184116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f6873-4156-4a2a-9d9b-389f0e0d9d64 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:20.196887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f6873-4156-4a2a-9d9b-389f0e0d9d64 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:20.198363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f6873-4156-4a2a-9d9b-389f0e0d9d64 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-a88f6873-4156-4a2a-9d9b-389f0e0d9d64 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:20.213343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:20.214348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:20.215292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:20.215292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:20.215292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:20.215292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:20.215292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:20.215292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:20.215292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:20.215292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:20.215292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:17:20.215292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:20.215292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 372/760] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 05:17:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:20.220929 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271e285-2882-4f51-ab57-d030f5bfdcc9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:20.270348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:20.372708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271e285-2882-4f51-ab57-d030f5bfdcc9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:20.416206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19ccb8-2dbb-48b4-b8b7-9a0050928e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:20.416824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19ccb8-2dbb-48b4-b8b7-9a0050928e98 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:20.417106 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19ccb8-2dbb-48b4-b8b7-9a0050928e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:20.417381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19ccb8-2dbb-48b4-b8b7-9a0050928e98 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:20.418184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19ccb8-2dbb-48b4-b8b7-9a0050928e98 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:20.418498 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19ccb8-2dbb-48b4-b8b7-9a0050928e98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:20.419929 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19ccb8-2dbb-48b4-b8b7-9a0050928e98 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:20.441852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 389/761] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 05:17:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:20.447415 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f6873-4156-4a2a-9d9b-389f0e0d9d64 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:20.607155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f6873-4156-4a2a-9d9b-389f0e0d9d64 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:20.657428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed073304-46bb-4ded-a995-a9c3291227f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:20.658641 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed073304-46bb-4ded-a995-a9c3291227f3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:20.658897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed073304-46bb-4ded-a995-a9c3291227f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:20.659187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed073304-46bb-4ded-a995-a9c3291227f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:20.670205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed073304-46bb-4ded-a995-a9c3291227f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:20.671681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed073304-46bb-4ded-a995-a9c3291227f3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ed073304-46bb-4ded-a995-a9c3291227f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:20.687043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:20.688060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:20.689051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:20.689051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:20.689051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:20.689051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 373/762] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:20.697197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19ccb8-2dbb-48b4-b8b7-9a0050928e98 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:20.818764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:20.842123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19ccb8-2dbb-48b4-b8b7-9a0050928e98 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:20.893035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc7e75-3c3c-40cf-b04b-2eb3467a0c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:20.893921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc7e75-3c3c-40cf-b04b-2eb3467a0c1a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:20.894202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc7e75-3c3c-40cf-b04b-2eb3467a0c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:20.894471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc7e75-3c3c-40cf-b04b-2eb3467a0c1a None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:20.895149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc7e75-3c3c-40cf-b04b-2eb3467a0c1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:20.895453 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc7e75-3c3c-40cf-b04b-2eb3467a0c1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:20.896907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc7e75-3c3c-40cf-b04b-2eb3467a0c1a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:20.914342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 390/763] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:20.920201 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed073304-46bb-4ded-a995-a9c3291227f3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:20.957253 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:21.035880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef89af-358c-4130-a3a3-1e16e174e108 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:21.036237 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef89af-358c-4130-a3a3-1e16e174e108 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:21.036457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef89af-358c-4130-a3a3-1e16e174e108 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:21.037784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 391/764] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:21.044181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02dccb-820f-4e3d-8267-7675fd6a8d83 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:21.044462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02dccb-820f-4e3d-8267-7675fd6a8d83 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:21.044727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02dccb-820f-4e3d-8267-7675fd6a8d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:21.077543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1a02dccb-820f-4e3d-8267-7675fd6a8d83 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:21.078746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1a02dccb-820f-4e3d-8267-7675fd6a8d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:21.105105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed073304-46bb-4ded-a995-a9c3291227f3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:21.181151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f59ae3-4334-4779-8e92-341c265196ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:21.181783 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f59ae3-4334-4779-8e92-341c265196ac None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:21.182059 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f59ae3-4334-4779-8e92-341c265196ac None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:21.182328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f59ae3-4334-4779-8e92-341c265196ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:21.193158 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f59ae3-4334-4779-8e92-341c265196ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:21.194627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f59ae3-4334-4779-8e92-341c265196ac None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-51f59ae3-4334-4779-8e92-341c265196ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:21.210849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:21.211758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:21.212704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:21.212704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:21.212704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:21.212704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:21.212704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:21.212704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:21.212704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:21.212704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:21.212704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 374/765] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:21.217306 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 392/766] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:21.218672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f59ae3-4334-4779-8e92-341c265196ac None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:21.351404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:21.380126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f59ae3-4334-4779-8e92-341c265196ac None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:21.428309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97128d-d4eb-427c-b661-2ccb0d436937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:21.429811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97128d-d4eb-427c-b661-2ccb0d436937 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:21.430596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97128d-d4eb-427c-b661-2ccb0d436937 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:21.431393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97128d-d4eb-427c-b661-2ccb0d436937 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:21.432829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97128d-d4eb-427c-b661-2ccb0d436937 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:21.433347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97128d-d4eb-427c-b661-2ccb0d436937 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:21.435028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97128d-d4eb-427c-b661-2ccb0d436937 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:21.455691 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 375/767] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:21.461610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02dccb-820f-4e3d-8267-7675fd6a8d83 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:21.543663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:21.572953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97128d-d4eb-427c-b661-2ccb0d436937 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:21.580186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02dccb-820f-4e3d-8267-7675fd6a8d83 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:21.616789 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c607d18-9352-4221-a1c6-a638092ead17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:21.617472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c607d18-9352-4221-a1c6-a638092ead17 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:21.617747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c607d18-9352-4221-a1c6-a638092ead17 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:21.618014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c607d18-9352-4221-a1c6-a638092ead17 None admin] PATH_INFO: `/v3/projects/092dd30ff8b541e2a45947504831998a/users/3bd5a9a09f034c05815c631a44d1cdd9/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:21.624376 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c607d18-9352-4221-a1c6-a638092ead17 None admin] RBAC: Authorizing `identity:create_grant(project_id=092dd30ff8b541e2a45947504831998a, user_id=3bd5a9a09f034c05815c631a44d1cdd9, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:21.625854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c607d18-9352-4221-a1c6-a638092ead17 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:21.654378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 393/768] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:17:21 2022] PUT /identity/v3/projects/092dd30ff8b541e2a45947504831998a/users/3bd5a9a09f034c05815c631a44d1cdd9/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:17:21.744255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cbaed-6876-4daa-8ef5-73ec11a58480 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:21.744429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cbaed-6876-4daa-8ef5-73ec11a58480 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:21.744702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cbaed-6876-4daa-8ef5-73ec11a58480 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:21.746469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 394/769] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:21.759812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97128d-d4eb-427c-b661-2ccb0d436937 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:21.764770 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8976750-c1cf-42a7-bd36-39525d41715b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:21.765051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8976750-c1cf-42a7-bd36-39525d41715b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:21.765323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8976750-c1cf-42a7-bd36-39525d41715b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:21.792599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d8976750-c1cf-42a7-bd36-39525d41715b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:21.793820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d8976750-c1cf-42a7-bd36-39525d41715b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:21.842021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d5d1f-83b4-4f88-8b6d-53d4687c956d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:21.842686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d5d1f-83b4-4f88-8b6d-53d4687c956d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:21.842967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d5d1f-83b4-4f88-8b6d-53d4687c956d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:21.843240 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d5d1f-83b4-4f88-8b6d-53d4687c956d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:21.843841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d5d1f-83b4-4f88-8b6d-53d4687c956d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:21.845731 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d5d1f-83b4-4f88-8b6d-53d4687c956d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:21.849804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-760d5d1f-83b4-4f88-8b6d-53d4687c956d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:21.861930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:21.863038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:21.863890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 376/770] 10.222.0.33 () {56 vars in 1129 bytes} [Sun Sep 11 05:17:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:21.869310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d5d1f-83b4-4f88-8b6d-53d4687c956d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:21.924180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 395/771] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:21.983268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0679a-e4ac-40bd-9b30-a7018405a2bb None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:21.983756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0679a-e4ac-40bd-9b30-a7018405a2bb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:21.984088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0679a-e4ac-40bd-9b30-a7018405a2bb None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:21.986976 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 396/772] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:21.990260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d5d1f-83b4-4f88-8b6d-53d4687c956d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:21.991300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a69d9-bcfa-48a0-910b-fb951175aa26 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:21.991685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a69d9-bcfa-48a0-910b-fb951175aa26 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:21.992113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a69d9-bcfa-48a0-910b-fb951175aa26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:22.015824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-240a69d9-bcfa-48a0-910b-fb951175aa26 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:22.015824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-240a69d9-bcfa-48a0-910b-fb951175aa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:22.038226 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b70b8-f2b1-4020-8019-80701eb87128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:22.039381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b70b8-f2b1-4020-8019-80701eb87128 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:22.040262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b70b8-f2b1-4020-8019-80701eb87128 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:22.040754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b70b8-f2b1-4020-8019-80701eb87128 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:22.041806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b70b8-f2b1-4020-8019-80701eb87128 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:22.042516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b70b8-f2b1-4020-8019-80701eb87128 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:22.044738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b70b8-f2b1-4020-8019-80701eb87128 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:22.051264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 377/773] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 11 05:17:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:22.061209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b70b8-f2b1-4020-8019-80701eb87128 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:22.103438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:22.122225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 397/774] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:22.133960 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:22.177478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b70b8-f2b1-4020-8019-80701eb87128 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:22.217630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88e93a-296f-48dd-92cc-03fa0c28d291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:22.218573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88e93a-296f-48dd-92cc-03fa0c28d291 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:22.219003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88e93a-296f-48dd-92cc-03fa0c28d291 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:22.219438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88e93a-296f-48dd-92cc-03fa0c28d291 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:22.220818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88e93a-296f-48dd-92cc-03fa0c28d291 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:22.221293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88e93a-296f-48dd-92cc-03fa0c28d291 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:22.222914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88e93a-296f-48dd-92cc-03fa0c28d291 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:22.228465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 378/775] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 11 05:17:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:22.234184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a69d9-bcfa-48a0-910b-fb951175aa26 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:22.246131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88e93a-296f-48dd-92cc-03fa0c28d291 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:22.351463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a69d9-bcfa-48a0-910b-fb951175aa26 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:22.405001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854acc08-be6c-42a8-8fa0-11ff50d6c1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:22.405613 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854acc08-be6c-42a8-8fa0-11ff50d6c1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:22.405888 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854acc08-be6c-42a8-8fa0-11ff50d6c1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:22.406155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854acc08-be6c-42a8-8fa0-11ff50d6c1d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:22.406855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854acc08-be6c-42a8-8fa0-11ff50d6c1d6 None admin] RBAC: Adding query filter params (interface=public, service_id=dd0f852ee94a44bb9d21e1e48ca9f8a8, region_id=RegionOne) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:22.407159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854acc08-be6c-42a8-8fa0-11ff50d6c1d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:22.408618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854acc08-be6c-42a8-8fa0-11ff50d6c1d6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:22.418787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88e93a-296f-48dd-92cc-03fa0c28d291 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:22.422201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 398/776] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 05:17:22 2022] GET /identity/v3/endpoints?service_id=dd0f852ee94a44bb9d21e1e48ca9f8a8&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:22.428194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854acc08-be6c-42a8-8fa0-11ff50d6c1d6 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:22.460382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:22.479441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22148f-d69b-4a02-8fa0-18ebc6214463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:22.480143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22148f-d69b-4a02-8fa0-18ebc6214463 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:22.480421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22148f-d69b-4a02-8fa0-18ebc6214463 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:22.480689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22148f-d69b-4a02-8fa0-18ebc6214463 None admin] PATH_INFO: `/v3/services/network` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:22.481268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22148f-d69b-4a02-8fa0-18ebc6214463 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:22.482633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22148f-d69b-4a02-8fa0-18ebc6214463 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-2d22148f-d69b-4a02-8fa0-18ebc6214463 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:22.497111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:22.498162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:22.499019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 379/777] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:22 2022] GET /identity/v3/services/network => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:22.499828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22148f-d69b-4a02-8fa0-18ebc6214463 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:22.534313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:22.586346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854acc08-be6c-42a8-8fa0-11ff50d6c1d6 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:22.621351 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22148f-d69b-4a02-8fa0-18ebc6214463 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:22.622573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a4be2-c004-49ec-bbaa-6af3daa9d8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:22.623259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a4be2-c004-49ec-bbaa-6af3daa9d8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:22.623497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a4be2-c004-49ec-bbaa-6af3daa9d8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:22.623758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a4be2-c004-49ec-bbaa-6af3daa9d8a0 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:22.624409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a4be2-c004-49ec-bbaa-6af3daa9d8a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:22.625789 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a4be2-c004-49ec-bbaa-6af3daa9d8a0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:22.640032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 399/778] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 05:17:22 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:22.640032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a4be2-c004-49ec-bbaa-6af3daa9d8a0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:22.691669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbe3bc-0769-4433-99b3-9431e1e2cc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:22.692651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbe3bc-0769-4433-99b3-9431e1e2cc5e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:22.693093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbe3bc-0769-4433-99b3-9431e1e2cc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:22.693592 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbe3bc-0769-4433-99b3-9431e1e2cc5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:22.710047 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbe3bc-0769-4433-99b3-9431e1e2cc5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-5bdbe3bc-0769-4433-99b3-9431e1e2cc5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:22.719454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:22.721197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:22.722068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:22.722068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:22.722068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:22.722068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:22.722068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:22.722068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:22.722068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:22.722068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:22.722068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:22.722068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:22.722068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:17:22.722068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:22.728031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 380/779] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:17:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:22.730615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbe3bc-0769-4433-99b3-9431e1e2cc5e None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:22.782194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a4be2-c004-49ec-bbaa-6af3daa9d8a0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:22.837560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:22.840692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:22.863217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04dbb5-3c53-4147-82a8-ad82d9514e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:22.863904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04dbb5-3c53-4147-82a8-ad82d9514e5b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:22.865390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04dbb5-3c53-4147-82a8-ad82d9514e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:22.865390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04dbb5-3c53-4147-82a8-ad82d9514e5b None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:22.865656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04dbb5-3c53-4147-82a8-ad82d9514e5b None admin] RBAC: Adding query filter params (name=network) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:22.865903 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04dbb5-3c53-4147-82a8-ad82d9514e5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:22.867308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04dbb5-3c53-4147-82a8-ad82d9514e5b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:22.881212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 400/780] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 05:17:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:22.882053 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52421004-02e8-43dc-beee-01136828a865 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:22.882333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52421004-02e8-43dc-beee-01136828a865 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:22.882601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52421004-02e8-43dc-beee-01136828a865 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:22.885908 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 401/781] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:22.888029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52421004-02e8-43dc-beee-01136828a865 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:22.923880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbe3bc-0769-4433-99b3-9431e1e2cc5e None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:22.972831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c33849-c374-43cd-be5a-1c4ce214d960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:22.974147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33849-c374-43cd-be5a-1c4ce214d960 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:22.974619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33849-c374-43cd-be5a-1c4ce214d960 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:22.975066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33849-c374-43cd-be5a-1c4ce214d960 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:22.976245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c33849-c374-43cd-be5a-1c4ce214d960 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:22.976746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c33849-c374-43cd-be5a-1c4ce214d960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:22.979025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c33849-c374-43cd-be5a-1c4ce214d960 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:22.990465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 381/782] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:17:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:22.991981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bdcc4-d26f-47ad-81fd-d894040104c7 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:22.992461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bdcc4-d26f-47ad-81fd-d894040104c7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:22.992921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bdcc4-d26f-47ad-81fd-d894040104c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:23.017616 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-196bdcc4-d26f-47ad-81fd-d894040104c7 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:23.019231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-196bdcc4-d26f-47ad-81fd-d894040104c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:23.036398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52421004-02e8-43dc-beee-01136828a865 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:23.081033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe9f14-3478-42a8-a8f4-12211c184a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:23.081907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe9f14-3478-42a8-a8f4-12211c184a8a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:23.082257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe9f14-3478-42a8-a8f4-12211c184a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:23.082592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe9f14-3478-42a8-a8f4-12211c184a8a None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:23.083547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe9f14-3478-42a8-a8f4-12211c184a8a None admin] RBAC: Adding query filter params (type=network) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:23.083926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe9f14-3478-42a8-a8f4-12211c184a8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:23.085810 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe9f14-3478-42a8-a8f4-12211c184a8a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:23.097991 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 402/783] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 05:17:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:23.098975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe9f14-3478-42a8-a8f4-12211c184a8a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:23.110345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 382/784] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:23.112286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bdcc4-d26f-47ad-81fd-d894040104c7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:23.163926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:23.216337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:23.232750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bdcc4-d26f-47ad-81fd-d894040104c7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:23.271398 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d0104-4b91-46b8-965a-34bc8564491b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:23.272351 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d0104-4b91-46b8-965a-34bc8564491b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:23.272807 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d0104-4b91-46b8-965a-34bc8564491b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:23.273281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d0104-4b91-46b8-965a-34bc8564491b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:23.274152 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d0104-4b91-46b8-965a-34bc8564491b None admin] RBAC: Adding query filter params (interface=public, service_id=491ff92f4377476290d089f8f33fba5d, region_id=RegionOne) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:23.274635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d0104-4b91-46b8-965a-34bc8564491b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:23.276202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d0104-4b91-46b8-965a-34bc8564491b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:23.284811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 383/785] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 05:17:23 2022] GET /identity/v3/endpoints?service_id=491ff92f4377476290d089f8f33fba5d&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:23.290877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d0104-4b91-46b8-965a-34bc8564491b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:23.334210 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe9f14-3478-42a8-a8f4-12211c184a8a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:23.388607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bd951-7508-48cd-9e56-4b8baa297423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:23.389561 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bd951-7508-48cd-9e56-4b8baa297423 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:23.390068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bd951-7508-48cd-9e56-4b8baa297423 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:23.390529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bd951-7508-48cd-9e56-4b8baa297423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:23.404078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bd951-7508-48cd-9e56-4b8baa297423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:23.405717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bd951-7508-48cd-9e56-4b8baa297423 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-bf7bd951-7508-48cd-9e56-4b8baa297423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:23.422082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:23.423004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:23.424675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:23.424675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:23.424675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:23.426779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 403/786] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:23.434540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bd951-7508-48cd-9e56-4b8baa297423 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:23.498821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d0104-4b91-46b8-965a-34bc8564491b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:23.510468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:23.552703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bd951-7508-48cd-9e56-4b8baa297423 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:23.553550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954d5ad-2554-4520-93f5-fbadeb5488fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:23.554585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:23.555035 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954d5ad-2554-4520-93f5-fbadeb5488fa None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:23.556338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954d5ad-2554-4520-93f5-fbadeb5488fa None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:23.556636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954d5ad-2554-4520-93f5-fbadeb5488fa None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:23.557259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954d5ad-2554-4520-93f5-fbadeb5488fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:23.559760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954d5ad-2554-4520-93f5-fbadeb5488fa None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:23.571127 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 384/787] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 05:17:23 2022] GET /identity/v3/services => generated 772 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:23.572328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954d5ad-2554-4520-93f5-fbadeb5488fa None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:23.646227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39458c-8748-4d47-b06b-e2f2fbed8981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:23.646891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39458c-8748-4d47-b06b-e2f2fbed8981 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:23.647106 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39458c-8748-4d47-b06b-e2f2fbed8981 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:23.647375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39458c-8748-4d47-b06b-e2f2fbed8981 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:23.648040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39458c-8748-4d47-b06b-e2f2fbed8981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:23.648348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39458c-8748-4d47-b06b-e2f2fbed8981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:23.649782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39458c-8748-4d47-b06b-e2f2fbed8981 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:23.682661 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 404/788] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:23.687949 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39458c-8748-4d47-b06b-e2f2fbed8981 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:23.709370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954d5ad-2554-4520-93f5-fbadeb5488fa None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:23.760153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877de74-aecc-4b27-98a5-63a04ab96685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:23.761704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877de74-aecc-4b27-98a5-63a04ab96685 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:23.763151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877de74-aecc-4b27-98a5-63a04ab96685 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:23.763654 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877de74-aecc-4b27-98a5-63a04ab96685 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:23.800788 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4877de74-aecc-4b27-98a5-63a04ab96685 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-4877de74-aecc-4b27-98a5-63a04ab96685 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:23.807313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:23.808364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:23.809207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:23.809207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:23.809207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:23.809207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:23.809207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:23.809207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:23.809207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:23.809207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:23.809207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:23.809207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:23.809207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:23.809207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:23.820044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 385/789] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:17:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:23.833038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877de74-aecc-4b27-98a5-63a04ab96685 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:23.872942 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:23.880509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39458c-8748-4d47-b06b-e2f2fbed8981 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:23.927028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce54d01-00c6-4eb9-9e25-889ed4ac7fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:23.928308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce54d01-00c6-4eb9-9e25-889ed4ac7fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:23.929634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce54d01-00c6-4eb9-9e25-889ed4ac7fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:23.930090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce54d01-00c6-4eb9-9e25-889ed4ac7fd9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:23.939756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce54d01-00c6-4eb9-9e25-889ed4ac7fd9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:23.941896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce54d01-00c6-4eb9-9e25-889ed4ac7fd9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-dce54d01-00c6-4eb9-9e25-889ed4ac7fd9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:23.959124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:23.960149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:23.961076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:23.961076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:23.961076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:23.961076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:23.961076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:23.961076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:23.961076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:23.961076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:23.961076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 05:17:23.961076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:23.962322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 405/790] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:17:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:23.969088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce54d01-00c6-4eb9-9e25-889ed4ac7fd9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:23.971289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877de74-aecc-4b27-98a5-63a04ab96685 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:24.015483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:24.040565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d3f4-b25f-4a2b-816c-70f9095c006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:24.043875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d3f4-b25f-4a2b-816c-70f9095c006b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:24.044303 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d3f4-b25f-4a2b-816c-70f9095c006b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:24.044591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d3f4-b25f-4a2b-816c-70f9095c006b None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:24.045314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1d3f4-b25f-4a2b-816c-70f9095c006b None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:24.045623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1d3f4-b25f-4a2b-816c-70f9095c006b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:24.049891 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1d3f4-b25f-4a2b-816c-70f9095c006b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:24.066879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 386/791] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:17:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:24.072480 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d3f4-b25f-4a2b-816c-70f9095c006b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:24.214072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:24.228891 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d3f4-b25f-4a2b-816c-70f9095c006b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:24.278650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35039f25-adea-4e23-9e87-ca03e97b6add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:24.279096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce54d01-00c6-4eb9-9e25-889ed4ac7fd9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:24.279553 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35039f25-adea-4e23-9e87-ca03e97b6add None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:24.279934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35039f25-adea-4e23-9e87-ca03e97b6add None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:24.280321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35039f25-adea-4e23-9e87-ca03e97b6add None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:24.294820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35039f25-adea-4e23-9e87-ca03e97b6add None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:24.297367 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35039f25-adea-4e23-9e87-ca03e97b6add None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-35039f25-adea-4e23-9e87-ca03e97b6add None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:24.317465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:24.318681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:24.319786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:24.319786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:24.319786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:24.320355 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 387/792] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:24.321483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672abf2-1a90-4a02-8feb-3cc687a3b79f None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:24.321766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672abf2-1a90-4a02-8feb-3cc687a3b79f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:24.322036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672abf2-1a90-4a02-8feb-3cc687a3b79f None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:24.323239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 388/793] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:24.328287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672abf2-1a90-4a02-8feb-3cc687a3b79f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:24.328452 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8562cac-d0a8-49b7-8c8c-825920892af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:24.329154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8562cac-d0a8-49b7-8c8c-825920892af2 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:24.329511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8562cac-d0a8-49b7-8c8c-825920892af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:24.329850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8562cac-d0a8-49b7-8c8c-825920892af2 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:24.330838 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8562cac-d0a8-49b7-8c8c-825920892af2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:24.331369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8562cac-d0a8-49b7-8c8c-825920892af2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:24.334149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8562cac-d0a8-49b7-8c8c-825920892af2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:24.359285 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 406/794] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 11 05:17:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:24.360927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b533d3-132c-4374-b454-2563ea7ee231 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:24.361285 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b533d3-132c-4374-b454-2563ea7ee231 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:24.361630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b533d3-132c-4374-b454-2563ea7ee231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:24.387135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a3b533d3-132c-4374-b454-2563ea7ee231 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:24.388919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a3b533d3-132c-4374-b454-2563ea7ee231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:24.469573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672abf2-1a90-4a02-8feb-3cc687a3b79f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:24.469761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 407/795] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:24.471434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b533d3-132c-4374-b454-2563ea7ee231 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:24.486235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:24.544736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398546b-a445-4e94-9cce-7f44723d3582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:24.545563 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398546b-a445-4e94-9cce-7f44723d3582 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:24.545944 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398546b-a445-4e94-9cce-7f44723d3582 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:24.546310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398546b-a445-4e94-9cce-7f44723d3582 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:24.547076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398546b-a445-4e94-9cce-7f44723d3582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:24.547510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398546b-a445-4e94-9cce-7f44723d3582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:24.549210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398546b-a445-4e94-9cce-7f44723d3582 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:24.578456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 389/796] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:24.586747 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398546b-a445-4e94-9cce-7f44723d3582 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:24.596096 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:24.607944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b533d3-132c-4374-b454-2563ea7ee231 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:24.647949 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822feadd-d4ba-4aec-a3d9-175b236defee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:24.648990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822feadd-d4ba-4aec-a3d9-175b236defee None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:24.649768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822feadd-d4ba-4aec-a3d9-175b236defee None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:24.650189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822feadd-d4ba-4aec-a3d9-175b236defee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:24.663002 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822feadd-d4ba-4aec-a3d9-175b236defee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:24.664735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822feadd-d4ba-4aec-a3d9-175b236defee None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-822feadd-d4ba-4aec-a3d9-175b236defee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:24.681804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:24.683009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:24.684131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:24.684131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:24.684131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:24.686068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 408/797] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:24.693116 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822feadd-d4ba-4aec-a3d9-175b236defee None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:24.767499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:24.787720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398546b-a445-4e94-9cce-7f44723d3582 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:24.819549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822feadd-d4ba-4aec-a3d9-175b236defee None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:24.837428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8dd2a2-1016-446b-8154-6517d3468c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:24.838541 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dd2a2-1016-446b-8154-6517d3468c04 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:24.838972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dd2a2-1016-446b-8154-6517d3468c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:24.839753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dd2a2-1016-446b-8154-6517d3468c04 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:24.850085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8dd2a2-1016-446b-8154-6517d3468c04 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:24.852734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8dd2a2-1016-446b-8154-6517d3468c04 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:24.859203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf37b02-0475-4c8f-a737-f9300e61363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:24.859959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf37b02-0475-4c8f-a737-f9300e61363c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:24.860388 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf37b02-0475-4c8f-a737-f9300e61363c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:24.860828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf37b02-0475-4c8f-a737-f9300e61363c None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:24.861578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf37b02-0475-4c8f-a737-f9300e61363c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:24.861994 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf37b02-0475-4c8f-a737-f9300e61363c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:24.863503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf37b02-0475-4c8f-a737-f9300e61363c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ff8dd2a2-1016-446b-8154-6517d3468c04 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:24.868830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:24.869945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:24.870909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:24.870909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:24.870909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:24.870909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:24.870909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:24.870909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:24.870909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:24.870909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:24.870909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 05:17:24.870909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:24.879555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 390/798] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 05:17:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:24.879555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8dd2a2-1016-446b-8154-6517d3468c04 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:24.885934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 409/799] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:24.887879 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf37b02-0475-4c8f-a737-f9300e61363c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:24.971758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:25.035839 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf37b02-0475-4c8f-a737-f9300e61363c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:25.071271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:25.075504 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de0803-5e35-4879-ba24-23ba135d54e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:25.076663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de0803-5e35-4879-ba24-23ba135d54e5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:25.077336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de0803-5e35-4879-ba24-23ba135d54e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:25.077857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de0803-5e35-4879-ba24-23ba135d54e5 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:25.079039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de0803-5e35-4879-ba24-23ba135d54e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:25.079442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de0803-5e35-4879-ba24-23ba135d54e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:25.081400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de0803-5e35-4879-ba24-23ba135d54e5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:25.101183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 410/800] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 11 05:17:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:25.105921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de0803-5e35-4879-ba24-23ba135d54e5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:25.132150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8dd2a2-1016-446b-8154-6517d3468c04 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:25.175204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b13bc-5642-4c28-bd19-7738ed48c441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:25.176126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b13bc-5642-4c28-bd19-7738ed48c441 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:25.176563 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b13bc-5642-4c28-bd19-7738ed48c441 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:25.177007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b13bc-5642-4c28-bd19-7738ed48c441 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:25.190401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b13bc-5642-4c28-bd19-7738ed48c441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-cb2b13bc-5642-4c28-bd19-7738ed48c441 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:25.196144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:25.197146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:25.197983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:25.197983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:25.197983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:25.197983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:25.197983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:25.197983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:25.197983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:25.197983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:25.197983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:25.197983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:25.197983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:17:25.197983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:25.197983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 391/801] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:17:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:25.199610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309505b1-c78a-4270-b232-11501ccaa606 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:25.199610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309505b1-c78a-4270-b232-11501ccaa606 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:25.199872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309505b1-c78a-4270-b232-11501ccaa606 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:25.201206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 392/802] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:25.206397 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309505b1-c78a-4270-b232-11501ccaa606 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:25.213551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de0803-5e35-4879-ba24-23ba135d54e5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:25.253896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf945f-5f38-4b61-860f-0bab97a68c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:25.254534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf945f-5f38-4b61-860f-0bab97a68c51 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:25.255029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf945f-5f38-4b61-860f-0bab97a68c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:25.255137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf945f-5f38-4b61-860f-0bab97a68c51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:25.262357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf945f-5f38-4b61-860f-0bab97a68c51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:25.263802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf945f-5f38-4b61-860f-0bab97a68c51 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e1cf945f-5f38-4b61-860f-0bab97a68c51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:25.275813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:25.276779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:25.277677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:25.277677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:25.277677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:25.277677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:25.277677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:25.277677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:25.277677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:25.277677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:25.277677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 411/803] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:25.277677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf945f-5f38-4b61-860f-0bab97a68c51 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:25.334480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:25.348481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309505b1-c78a-4270-b232-11501ccaa606 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:25.373208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:25.385066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4d77c-99ea-4abe-a0cc-c88dd1d6d0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:25.385689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4d77c-99ea-4abe-a0cc-c88dd1d6d0db None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:25.385967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4d77c-99ea-4abe-a0cc-c88dd1d6d0db None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:25.386265 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4d77c-99ea-4abe-a0cc-c88dd1d6d0db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:25.396634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4d77c-99ea-4abe-a0cc-c88dd1d6d0db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:25.398072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4d77c-99ea-4abe-a0cc-c88dd1d6d0db None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:25.406754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf945f-5f38-4b61-860f-0bab97a68c51 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-3fb4d77c-99ea-4abe-a0cc-c88dd1d6d0db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:25.412549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:25.413552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:25.419259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:25.419259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:25.419259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:25.419259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 393/804] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:25.419259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eba3c-51e2-4cc6-b665-15041f5fc39a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:25.419259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eba3c-51e2-4cc6-b665-15041f5fc39a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:25.419259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eba3c-51e2-4cc6-b665-15041f5fc39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:25.434190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-441eba3c-51e2-4cc6-b665-15041f5fc39a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:25.435372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-441eba3c-51e2-4cc6-b665-15041f5fc39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:25.444751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c20df-54da-46f2-adce-cb0aded8d02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:25.445377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c20df-54da-46f2-adce-cb0aded8d02f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:25.445656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c20df-54da-46f2-adce-cb0aded8d02f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:25.445926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c20df-54da-46f2-adce-cb0aded8d02f None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:25.446578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c20df-54da-46f2-adce-cb0aded8d02f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:25.446877 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c20df-54da-46f2-adce-cb0aded8d02f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:25.448251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c20df-54da-46f2-adce-cb0aded8d02f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:25.460945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 412/805] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:17:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:25.460945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c20df-54da-46f2-adce-cb0aded8d02f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:25.512570 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 394/806] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:25.513618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441eba3c-51e2-4cc6-b665-15041f5fc39a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:25.578354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c20df-54da-46f2-adce-cb0aded8d02f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:25.580902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:25.619480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59176777-dc71-413e-a8b8-fe73976c6320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:25.620150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59176777-dc71-413e-a8b8-fe73976c6320 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:25.620451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59176777-dc71-413e-a8b8-fe73976c6320 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:25.620803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59176777-dc71-413e-a8b8-fe73976c6320 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:25.622004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59176777-dc71-413e-a8b8-fe73976c6320 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:25.622569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59176777-dc71-413e-a8b8-fe73976c6320 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:25.624163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59176777-dc71-413e-a8b8-fe73976c6320 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:25.631618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441eba3c-51e2-4cc6-b665-15041f5fc39a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:25.638292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 413/807] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:25.639131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59176777-dc71-413e-a8b8-fe73976c6320 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:25.643975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:25.687836 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3bc22-85b7-4c44-bfdd-69da8be6a70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:25.689270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3bc22-85b7-4c44-bfdd-69da8be6a70b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:25.689703 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3bc22-85b7-4c44-bfdd-69da8be6a70b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:25.690710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3bc22-85b7-4c44-bfdd-69da8be6a70b None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:25.691511 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3bc22-85b7-4c44-bfdd-69da8be6a70b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:25.692489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3bc22-85b7-4c44-bfdd-69da8be6a70b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:25.694759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3bc22-85b7-4c44-bfdd-69da8be6a70b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:25.716374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 395/808] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:25.717199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3bc22-85b7-4c44-bfdd-69da8be6a70b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:25.764710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59176777-dc71-413e-a8b8-fe73976c6320 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:25.814526 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e37261-4d17-4aa1-b4f8-c71185a8a45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:25.815442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e37261-4d17-4aa1-b4f8-c71185a8a45a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:25.815725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e37261-4d17-4aa1-b4f8-c71185a8a45a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:25.816063 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e37261-4d17-4aa1-b4f8-c71185a8a45a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:25.827203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e37261-4d17-4aa1-b4f8-c71185a8a45a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:25.828836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e37261-4d17-4aa1-b4f8-c71185a8a45a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d9e37261-4d17-4aa1-b4f8-c71185a8a45a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:25.848559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:25.849676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:25.850588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:25.850588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:25.850588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:25.850588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 414/809] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:25.850827 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e37261-4d17-4aa1-b4f8-c71185a8a45a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:25.855266 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3bc22-85b7-4c44-bfdd-69da8be6a70b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:25.871239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:25.893552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67021d91-2854-4fd5-9589-db420ef3c49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:25.894305 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67021d91-2854-4fd5-9589-db420ef3c49d None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:25.894655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67021d91-2854-4fd5-9589-db420ef3c49d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:25.894863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67021d91-2854-4fd5-9589-db420ef3c49d None admin] PATH_INFO: `/v3/projects/092dd30ff8b541e2a45947504831998a/users/0a726949e8cc4cfdb25aa78f2761cff5/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:25.897258 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67021d91-2854-4fd5-9589-db420ef3c49d None admin] RBAC: Authorizing `identity:create_grant(project_id=092dd30ff8b541e2a45947504831998a, user_id=0a726949e8cc4cfdb25aa78f2761cff5, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:25.898700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67021d91-2854-4fd5-9589-db420ef3c49d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:25.920032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 396/810] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:17:25 2022] PUT /identity/v3/projects/092dd30ff8b541e2a45947504831998a/users/0a726949e8cc4cfdb25aa78f2761cff5/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:17:25.920032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67021d91-2854-4fd5-9589-db420ef3c49d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:25.953181 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:25.984224 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e37261-4d17-4aa1-b4f8-c71185a8a45a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:26.035096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad4381-5823-4099-a729-41b6704157dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:26.036092 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad4381-5823-4099-a729-41b6704157dd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:26.036598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad4381-5823-4099-a729-41b6704157dd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:26.036870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad4381-5823-4099-a729-41b6704157dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:26.046054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad4381-5823-4099-a729-41b6704157dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:26.047605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad4381-5823-4099-a729-41b6704157dd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:26.057460 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67021d91-2854-4fd5-9589-db420ef3c49d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-bdad4381-5823-4099-a729-41b6704157dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:26.062336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:26.063333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:26.064570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:26.064570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:26.064570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:26.064570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:26.064570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:26.064570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:26.064570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:26.064570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:26.064570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 415/811] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:26.065223 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad4381-5823-4099-a729-41b6704157dd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:26.104607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf77d3a-1cfb-45ac-9241-e07438111916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:26.105783 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf77d3a-1cfb-45ac-9241-e07438111916 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:26.106290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf77d3a-1cfb-45ac-9241-e07438111916 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:26.106832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf77d3a-1cfb-45ac-9241-e07438111916 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:26.107621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf77d3a-1cfb-45ac-9241-e07438111916 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:26.109455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf77d3a-1cfb-45ac-9241-e07438111916 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-3bf77d3a-1cfb-45ac-9241-e07438111916 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:26.120852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:26.121993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 05:17:26.122847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:26.125650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 397/812] 10.222.0.33 () {56 vars in 1129 bytes} [Sun Sep 11 05:17:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:26.127546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf77d3a-1cfb-45ac-9241-e07438111916 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:26.157431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:26.193699 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad4381-5823-4099-a729-41b6704157dd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:26.230819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d902d63-617d-4c38-8006-860bf0332b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:26.231421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d902d63-617d-4c38-8006-860bf0332b6b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:26.231694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d902d63-617d-4c38-8006-860bf0332b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:26.231975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d902d63-617d-4c38-8006-860bf0332b6b None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:26.232627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d902d63-617d-4c38-8006-860bf0332b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:26.232952 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d902d63-617d-4c38-8006-860bf0332b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:26.234303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d902d63-617d-4c38-8006-860bf0332b6b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:26.239750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:26.246860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 416/813] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:26.251383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d902d63-617d-4c38-8006-860bf0332b6b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:26.256794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf77d3a-1cfb-45ac-9241-e07438111916 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:26.294329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b13836-53c2-4e46-8ac7-2a637a500e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:26.295122 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b13836-53c2-4e46-8ac7-2a637a500e1e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:26.295405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b13836-53c2-4e46-8ac7-2a637a500e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:26.295677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b13836-53c2-4e46-8ac7-2a637a500e1e None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:26.296404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b13836-53c2-4e46-8ac7-2a637a500e1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:26.296708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b13836-53c2-4e46-8ac7-2a637a500e1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:26.298190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b13836-53c2-4e46-8ac7-2a637a500e1e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:26.315819 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 398/814] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:26.315819 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169e863-959a-4759-9bdf-a9158b0c7a63 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:26.315819 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169e863-959a-4759-9bdf-a9158b0c7a63 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:26.315819 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169e863-959a-4759-9bdf-a9158b0c7a63 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:26.315819 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 399/815] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:26.315819 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169e863-959a-4759-9bdf-a9158b0c7a63 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:26.362476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d902d63-617d-4c38-8006-860bf0332b6b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:26.397847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1574df2-58ef-4911-9109-3a98984c957d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:26.398371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1574df2-58ef-4911-9109-3a98984c957d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:26.398649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1574df2-58ef-4911-9109-3a98984c957d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:26.398921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1574df2-58ef-4911-9109-3a98984c957d None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:26.399546 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1574df2-58ef-4911-9109-3a98984c957d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:26.399849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1574df2-58ef-4911-9109-3a98984c957d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:26.401255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1574df2-58ef-4911-9109-3a98984c957d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:26.402307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:26.406024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 417/816] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 11 05:17:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:26.406834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1574df2-58ef-4911-9109-3a98984c957d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:26.433225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169e863-959a-4759-9bdf-a9158b0c7a63 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:26.469937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9b54c-8bf7-41cf-a88c-02e3b9ed48d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:26.471380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9b54c-8bf7-41cf-a88c-02e3b9ed48d5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:26.471665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9b54c-8bf7-41cf-a88c-02e3b9ed48d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:26.471938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9b54c-8bf7-41cf-a88c-02e3b9ed48d5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:26.479073 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9b54c-8bf7-41cf-a88c-02e3b9ed48d5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:26.480559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9b54c-8bf7-41cf-a88c-02e3b9ed48d5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-a8c9b54c-8bf7-41cf-a88c-02e3b9ed48d5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:26.492189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:26.493381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:26.494521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:26.494521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:26.494521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:26.494521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:26.494521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:26.494521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:26.494521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:26.494521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:26.494521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:17:26.494521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:26.500037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 400/817] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 05:17:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:26.500037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245da7c-5607-41c2-b50a-1670e4ed775b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:26.500037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245da7c-5607-41c2-b50a-1670e4ed775b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:26.500037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245da7c-5607-41c2-b50a-1670e4ed775b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:26.515891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1574df2-58ef-4911-9109-3a98984c957d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:26.518571 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f245da7c-5607-41c2-b50a-1670e4ed775b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:26.519759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f245da7c-5607-41c2-b50a-1670e4ed775b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:26.522631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:26.553932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee8c05-ed87-4f36-97c3-f9f6972c6db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:26.554652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee8c05-ed87-4f36-97c3-f9f6972c6db1 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:26.554933 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee8c05-ed87-4f36-97c3-f9f6972c6db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:26.555205 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee8c05-ed87-4f36-97c3-f9f6972c6db1 None admin] PATH_INFO: `/v3/projects/092dd30ff8b541e2a45947504831998a/users/afd0a6d3b2194604a46767cfde12097b/roles/7a06d87f01d940b38065f296fa63d0f7` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:26.573232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee8c05-ed87-4f36-97c3-f9f6972c6db1 None admin] RBAC: Authorizing `identity:create_grant(project_id=092dd30ff8b541e2a45947504831998a, user_id=afd0a6d3b2194604a46767cfde12097b, role_id=7a06d87f01d940b38065f296fa63d0f7)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:26.574796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee8c05-ed87-4f36-97c3-f9f6972c6db1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:26.596044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 418/818] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:17:26 2022] PUT /identity/v3/projects/092dd30ff8b541e2a45947504831998a/users/afd0a6d3b2194604a46767cfde12097b/roles/7a06d87f01d940b38065f296fa63d0f7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:17:26.596044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee8c05-ed87-4f36-97c3-f9f6972c6db1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:26.602737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 401/819] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:26.603646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245da7c-5607-41c2-b50a-1670e4ed775b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:26.704440 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:26.783348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245da7c-5607-41c2-b50a-1670e4ed775b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:26.827047 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771350e-801a-41cf-b8a5-0cd62df13df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:26.827661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771350e-801a-41cf-b8a5-0cd62df13df8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:26.827942 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771350e-801a-41cf-b8a5-0cd62df13df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:26.828271 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771350e-801a-41cf-b8a5-0cd62df13df8 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:26.829009 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771350e-801a-41cf-b8a5-0cd62df13df8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:26.829832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771350e-801a-41cf-b8a5-0cd62df13df8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:26.831652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771350e-801a-41cf-b8a5-0cd62df13df8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:26.842212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:26.852457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee8c05-ed87-4f36-97c3-f9f6972c6db1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:26.854488 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 402/820] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 05:17:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:26.859918 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771350e-801a-41cf-b8a5-0cd62df13df8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:26.901940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57df817-2fc1-46b0-8cc5-9c40185040dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:26.902952 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57df817-2fc1-46b0-8cc5-9c40185040dd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:26.903445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57df817-2fc1-46b0-8cc5-9c40185040dd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:26.903916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57df817-2fc1-46b0-8cc5-9c40185040dd None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:26.904805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57df817-2fc1-46b0-8cc5-9c40185040dd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:26.905382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57df817-2fc1-46b0-8cc5-9c40185040dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:26.907580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57df817-2fc1-46b0-8cc5-9c40185040dd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:26.920292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 419/821] 10.222.0.33 () {56 vars in 1124 bytes} [Sun Sep 11 05:17:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:26.924258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57df817-2fc1-46b0-8cc5-9c40185040dd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:26.979865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771350e-801a-41cf-b8a5-0cd62df13df8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:27.019832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f4071-bd9c-49ee-871f-b7c28aba95c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:27.020913 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f4071-bd9c-49ee-871f-b7c28aba95c8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:27.021203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f4071-bd9c-49ee-871f-b7c28aba95c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:27.021471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f4071-bd9c-49ee-871f-b7c28aba95c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:27.053394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57df817-2fc1-46b0-8cc5-9c40185040dd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:27.068470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:27.095109 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c50d45-cecf-42c6-b409-37262816c84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:27.095721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50d45-cecf-42c6-b409-37262816c84e None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:27.096027 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50d45-cecf-42c6-b409-37262816c84e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:27.096292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50d45-cecf-42c6-b409-37262816c84e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:27.096956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c50d45-cecf-42c6-b409-37262816c84e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:27.098317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c50d45-cecf-42c6-b409-37262816c84e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:27.098697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:27.103232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f4071-bd9c-49ee-871f-b7c28aba95c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:27.105123 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f4071-bd9c-49ee-871f-b7c28aba95c8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:27.132020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 420/822] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:17:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:17:27.132020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c50d45-cecf-42c6-b409-37262816c84e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:27.291008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 403/823] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:17:27.297698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f4071-bd9c-49ee-871f-b7c28aba95c8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:27.317239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c50d45-cecf-42c6-b409-37262816c84e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:27.359901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e8889-6aed-4229-87e3-b9eb66665925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:27.360812 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e8889-6aed-4229-87e3-b9eb66665925 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:27.361240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e8889-6aed-4229-87e3-b9eb66665925 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:27.361603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e8889-6aed-4229-87e3-b9eb66665925 None admin] PATH_INFO: `/v3/services/network` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:27.362192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e8889-6aed-4229-87e3-b9eb66665925 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:27.363558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e8889-6aed-4229-87e3-b9eb66665925 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-7f1e8889-6aed-4229-87e3-b9eb66665925 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:27.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:27.377020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:27.377882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 421/824] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:27 2022] GET /identity/v3/services/network => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:27.386754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e8889-6aed-4229-87e3-b9eb66665925 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:27.422468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:27.438662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:27.443645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f4071-bd9c-49ee-871f-b7c28aba95c8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:27.491239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c68122-2750-4a40-868e-c761f8e1ab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:27.492625 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c68122-2750-4a40-868e-c761f8e1ab47 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:27.493092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c68122-2750-4a40-868e-c761f8e1ab47 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:27.493718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c68122-2750-4a40-868e-c761f8e1ab47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:27.507401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c68122-2750-4a40-868e-c761f8e1ab47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:27.509684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c68122-2750-4a40-868e-c761f8e1ab47 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:27.522074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e8889-6aed-4229-87e3-b9eb66665925 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-53c68122-2750-4a40-868e-c761f8e1ab47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:27.527774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:27.528735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:27.529639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:27.529639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:27.529639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:27.532597 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 404/825] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:27.542990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c68122-2750-4a40-868e-c761f8e1ab47 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:27.588928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fa5f7-10db-49af-88da-81dc3a7f4ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:27.589533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fa5f7-10db-49af-88da-81dc3a7f4ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:27.589809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fa5f7-10db-49af-88da-81dc3a7f4ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:27.590129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fa5f7-10db-49af-88da-81dc3a7f4ef7 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:27.590756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fa5f7-10db-49af-88da-81dc3a7f4ef7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:27.591106 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fa5f7-10db-49af-88da-81dc3a7f4ef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:27.592558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fa5f7-10db-49af-88da-81dc3a7f4ef7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:27.604225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 422/826] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 05:17:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:27.609097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fa5f7-10db-49af-88da-81dc3a7f4ef7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:27.698976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c68122-2750-4a40-868e-c761f8e1ab47 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:27.772714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192315e3-fad4-4851-a421-0ce364b264a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:27.773334 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192315e3-fad4-4851-a421-0ce364b264a1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:27.773619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192315e3-fad4-4851-a421-0ce364b264a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:27.773892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192315e3-fad4-4851-a421-0ce364b264a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:27.774544 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192315e3-fad4-4851-a421-0ce364b264a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:27.774850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192315e3-fad4-4851-a421-0ce364b264a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:27.776249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192315e3-fad4-4851-a421-0ce364b264a1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:27.779522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:27.796975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fa5f7-10db-49af-88da-81dc3a7f4ef7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:27.807330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 405/827] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:27.814849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192315e3-fad4-4851-a421-0ce364b264a1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:27.842792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec63d6-adec-44a9-94bb-2dc24bfef9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:27.843405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec63d6-adec-44a9-94bb-2dc24bfef9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:27.843685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec63d6-adec-44a9-94bb-2dc24bfef9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:27.843953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec63d6-adec-44a9-94bb-2dc24bfef9c2 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:27.844622 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec63d6-adec-44a9-94bb-2dc24bfef9c2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:27.844959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec63d6-adec-44a9-94bb-2dc24bfef9c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:27.848245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec63d6-adec-44a9-94bb-2dc24bfef9c2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:27.849756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:27.854700 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 423/828] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 05:17:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:27.861550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec63d6-adec-44a9-94bb-2dc24bfef9c2 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:28.027293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec63d6-adec-44a9-94bb-2dc24bfef9c2 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:28.041956 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192315e3-fad4-4851-a421-0ce364b264a1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:28.072740 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366780b-19f9-4458-853b-bdcbe804b24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:28.073366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366780b-19f9-4458-853b-bdcbe804b24e None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:28.073649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366780b-19f9-4458-853b-bdcbe804b24e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:28.073918 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366780b-19f9-4458-853b-bdcbe804b24e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:28.074599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366780b-19f9-4458-853b-bdcbe804b24e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:28.075970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366780b-19f9-4458-853b-bdcbe804b24e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:28.092666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4267e2-67f4-47f7-b8c3-04c0e110ba66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:28.093684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4267e2-67f4-47f7-b8c3-04c0e110ba66 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:28.094170 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4267e2-67f4-47f7-b8c3-04c0e110ba66 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:28.094698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4267e2-67f4-47f7-b8c3-04c0e110ba66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:28.102872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4267e2-67f4-47f7-b8c3-04c0e110ba66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:28.104767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4267e2-67f4-47f7-b8c3-04c0e110ba66 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ea4267e2-67f4-47f7-b8c3-04c0e110ba66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:28.117279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:28.118302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:28.119202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:28.119202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:28.119202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:28.119202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:28.119202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:28.119202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:28.119202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:28.119202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:28.121927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 406/829] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:28.129489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4267e2-67f4-47f7-b8c3-04c0e110ba66 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:28.147984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 424/830] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:17:27 2022] POST /identity/v3/endpoints => generated 337 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:17:28.212732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:28.249061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4267e2-67f4-47f7-b8c3-04c0e110ba66 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:28.295855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae06986-059e-42dd-98bb-b537f4a624f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:28.296902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae06986-059e-42dd-98bb-b537f4a624f2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:28.297423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae06986-059e-42dd-98bb-b537f4a624f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:28.297931 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae06986-059e-42dd-98bb-b537f4a624f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:28.299550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae06986-059e-42dd-98bb-b537f4a624f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:28.300158 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae06986-059e-42dd-98bb-b537f4a624f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:28.301981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae06986-059e-42dd-98bb-b537f4a624f2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:28.318288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 407/831] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:28.328132 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366780b-19f9-4458-853b-bdcbe804b24e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:28.355747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:28.456474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366780b-19f9-4458-853b-bdcbe804b24e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:28.491180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355af8f-b07a-4054-bd2c-e83f796770b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:28.491857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355af8f-b07a-4054-bd2c-e83f796770b0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:28.492168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355af8f-b07a-4054-bd2c-e83f796770b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:28.492438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355af8f-b07a-4054-bd2c-e83f796770b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:28.493165 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355af8f-b07a-4054-bd2c-e83f796770b0 None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=3bd5a9a09f034c05815c631a44d1cdd9) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:28.493485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355af8f-b07a-4054-bd2c-e83f796770b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:28.494911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355af8f-b07a-4054-bd2c-e83f796770b0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:28.501126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 425/832] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:17:28 2022] GET /identity/v3/role_assignments?user.id=3bd5a9a09f034c05815c631a44d1cdd9&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:28.512532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd5c0d-85d4-4b15-aa75-ec87aac159da None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:28.512837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd5c0d-85d4-4b15-aa75-ec87aac159da None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:28.513100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd5c0d-85d4-4b15-aa75-ec87aac159da None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:28.514822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 408/833] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:28.529993 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9226f2-b8ae-4c65-81e3-9c9fb2544cfb None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:28.530367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9226f2-b8ae-4c65-81e3-9c9fb2544cfb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:28.530711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9226f2-b8ae-4c65-81e3-9c9fb2544cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:28.550632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ee9226f2-b8ae-4c65-81e3-9c9fb2544cfb None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:28.551928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ee9226f2-b8ae-4c65-81e3-9c9fb2544cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:28.629142 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 426/834] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:28.929708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd5c0d-85d4-4b15-aa75-ec87aac159da None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:29.041732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd5c0d-85d4-4b15-aa75-ec87aac159da None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:29.069449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:29.080150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3891fc4-897d-4e61-b595-6289a0ffcd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:29.080904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3891fc4-897d-4e61-b595-6289a0ffcd27 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:29.081191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3891fc4-897d-4e61-b595-6289a0ffcd27 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:29.081468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3891fc4-897d-4e61-b595-6289a0ffcd27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:29.092409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3891fc4-897d-4e61-b595-6289a0ffcd27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c3891fc4-897d-4e61-b595-6289a0ffcd27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:29.097416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:29.098366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:29.099212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:29.099212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:29.099212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:29.099212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:29.099212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:29.099212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:29.099212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:29.099212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:29.099212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:29.099212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:29.099212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:17:29.099212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:29.099212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 409/835] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:17:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:29.104289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9226f2-b8ae-4c65-81e3-9c9fb2544cfb None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:29.120364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:29.214773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9226f2-b8ae-4c65-81e3-9c9fb2544cfb None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:29.250520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c169b0-4838-4c04-85b1-69cba8228b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:29.251196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c169b0-4838-4c04-85b1-69cba8228b97 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:29.251660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c169b0-4838-4c04-85b1-69cba8228b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:29.252040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c169b0-4838-4c04-85b1-69cba8228b97 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:29.252798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c169b0-4838-4c04-85b1-69cba8228b97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:29.253109 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c169b0-4838-4c04-85b1-69cba8228b97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:29.254735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c169b0-4838-4c04-85b1-69cba8228b97 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:29.263916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 427/836] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:17:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:29.273116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3891fc4-897d-4e61-b595-6289a0ffcd27 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:29.287718 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528717fa-3ebc-4751-9553-1102a0f7d352 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:29.288055 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528717fa-3ebc-4751-9553-1102a0f7d352 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:29.288410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528717fa-3ebc-4751-9553-1102a0f7d352 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:29.290024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 428/837] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:29.296672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7b9be-063b-4ed9-b486-d793076fe18f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:29.296953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7b9be-063b-4ed9-b486-d793076fe18f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:29.297224 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7b9be-063b-4ed9-b486-d793076fe18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:29.319211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d1c7b9be-063b-4ed9-b486-d793076fe18f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:29.320258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d1c7b9be-063b-4ed9-b486-d793076fe18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:29.386361 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3891fc4-897d-4e61-b595-6289a0ffcd27 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:29.402340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:29.412169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 429/838] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:29.429911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7b546-a8d1-409b-9167-8c97bc377675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:29.430890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7b546-a8d1-409b-9167-8c97bc377675 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:29.431402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7b546-a8d1-409b-9167-8c97bc377675 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:29.431903 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7b546-a8d1-409b-9167-8c97bc377675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:29.442723 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7b546-a8d1-409b-9167-8c97bc377675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:29.444618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7b546-a8d1-409b-9167-8c97bc377675 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-48f7b546-a8d1-409b-9167-8c97bc377675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:29.459923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:29.460937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:29.461839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:29.461839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:29.461839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:29.463829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 410/839] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:29.469171 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7b9be-063b-4ed9-b486-d793076fe18f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:29.583247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7b9be-063b-4ed9-b486-d793076fe18f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:29.619793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d1943-293e-4a43-a31c-9ef2a6a7fb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:29.620430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d1943-293e-4a43-a31c-9ef2a6a7fb9d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:29.620710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d1943-293e-4a43-a31c-9ef2a6a7fb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:29.620981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d1943-293e-4a43-a31c-9ef2a6a7fb9d None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:29.621627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d1943-293e-4a43-a31c-9ef2a6a7fb9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:29.621932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d1943-293e-4a43-a31c-9ef2a6a7fb9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:29.623299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d1943-293e-4a43-a31c-9ef2a6a7fb9d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:29.635853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 430/840] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:29.640923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7b546-a8d1-409b-9167-8c97bc377675 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:29.685038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:29.694330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835cade-0cc1-475f-8312-76d8aca8ae8f None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:29.694628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835cade-0cc1-475f-8312-76d8aca8ae8f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:29.694898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835cade-0cc1-475f-8312-76d8aca8ae8f None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:29.696592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 431/841] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:29.702704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9d83-52f9-4513-bf1f-7cce1d34a985 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:29.703000 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9d83-52f9-4513-bf1f-7cce1d34a985 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:29.703280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9d83-52f9-4513-bf1f-7cce1d34a985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:29.724882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f4cd9d83-52f9-4513-bf1f-7cce1d34a985 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:29.725995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f4cd9d83-52f9-4513-bf1f-7cce1d34a985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:29.754671 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7b546-a8d1-409b-9167-8c97bc377675 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:29.786176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:29.798665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab7e8a-c720-4523-ae91-921b729626de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:29.800481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab7e8a-c720-4523-ae91-921b729626de None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:29.800985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab7e8a-c720-4523-ae91-921b729626de None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:29.801503 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab7e8a-c720-4523-ae91-921b729626de None admin] PATH_INFO: `/v3/users/swift` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:29.804865 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 432/842] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:29.805720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd9d83-52f9-4513-bf1f-7cce1d34a985 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:29.812702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab7e8a-c720-4523-ae91-921b729626de None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:29.814981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab7e8a-c720-4523-ae91-921b729626de None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-90ab7e8a-c720-4523-ae91-921b729626de None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:29.830255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:29.831551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:29.837896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:29.837896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:29.837896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:29.837896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:29.837896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:29.837896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:29.837896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:29.837896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:29.837896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 05:17:29.837896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:29.837896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 411/843] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:17:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:29.838554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab7e8a-c720-4523-ae91-921b729626de None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:29.948872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd9d83-52f9-4513-bf1f-7cce1d34a985 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:29.959248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:29.963442 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab7e8a-c720-4523-ae91-921b729626de None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:29.998477 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4e532-1ece-49c7-8151-ec0ee4c7c0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:29.999350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c113d-863b-4712-b711-bf814e2956f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:29.999545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4e532-1ece-49c7-8151-ec0ee4c7c0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:29.999959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4e532-1ece-49c7-8151-ec0ee4c7c0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:30.000347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c113d-863b-4712-b711-bf814e2956f0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:30.000557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4e532-1ece-49c7-8151-ec0ee4c7c0b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:30.000812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c113d-863b-4712-b711-bf814e2956f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:30.001310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c113d-863b-4712-b711-bf814e2956f0 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:30.002180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c113d-863b-4712-b711-bf814e2956f0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:30.002584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c113d-863b-4712-b711-bf814e2956f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:30.004467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c113d-863b-4712-b711-bf814e2956f0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:30.016527 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4e532-1ece-49c7-8151-ec0ee4c7c0b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-c9d4e532-1ece-49c7-8151-ec0ee4c7c0b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:30.023081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:30.024257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:30.025291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:30.025291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:30.025291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:30.025291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:30.025291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:30.025291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:30.025291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:30.025291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:30.025291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:30.025291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:30.025291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:17:30.025291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:30.026839 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 433/844] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:17:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:30.033251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4e532-1ece-49c7-8151-ec0ee4c7c0b5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:30.033915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 412/845] 10.222.0.33 () {56 vars in 1135 bytes} [Sun Sep 11 05:17:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:30.035616 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c113d-863b-4712-b711-bf814e2956f0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:30.085761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:30.142954 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4e532-1ece-49c7-8151-ec0ee4c7c0b5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:30.153666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c113d-863b-4712-b711-bf814e2956f0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:30.179532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3a7e9-1efa-4123-8853-b44ec60533d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:30.180160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3a7e9-1efa-4123-8853-b44ec60533d9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:30.180436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3a7e9-1efa-4123-8853-b44ec60533d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:30.180701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3a7e9-1efa-4123-8853-b44ec60533d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:30.181360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3a7e9-1efa-4123-8853-b44ec60533d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:30.181655 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3a7e9-1efa-4123-8853-b44ec60533d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:30.183007 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa3a7e9-1efa-4123-8853-b44ec60533d9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:30.191484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3ee90-9dfd-431a-b89d-e4d08a70d1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:30.192028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 434/846] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:17:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:30.192754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3a7e9-1efa-4123-8853-b44ec60533d9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:30.196808 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3ee90-9dfd-431a-b89d-e4d08a70d1dc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:30.196939 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3ee90-9dfd-431a-b89d-e4d08a70d1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:30.197221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3ee90-9dfd-431a-b89d-e4d08a70d1dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:30.224137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:30.269246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3ee90-9dfd-431a-b89d-e4d08a70d1dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:30.270592 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3ee90-9dfd-431a-b89d-e4d08a70d1dc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:30.302222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3a7e9-1efa-4123-8853-b44ec60533d9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:30.313101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:30.338907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab0093-4730-4ce3-b1f0-e8c0d6a9b80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:30.339540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab0093-4730-4ce3-b1f0-e8c0d6a9b80d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:30.339842 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab0093-4730-4ce3-b1f0-e8c0d6a9b80d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:30.340131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab0093-4730-4ce3-b1f0-e8c0d6a9b80d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:30.340751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab0093-4730-4ce3-b1f0-e8c0d6a9b80d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:30.342097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab0093-4730-4ce3-b1f0-e8c0d6a9b80d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-2dab0093-4730-4ce3-b1f0-e8c0d6a9b80d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:30.351543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:30.356332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:30.357192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 435/847] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:17:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:30.357975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab0093-4730-4ce3-b1f0-e8c0d6a9b80d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:30.383821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 413/848] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:30 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 05:17:30.384899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3ee90-9dfd-431a-b89d-e4d08a70d1dc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:30.449116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:30.465471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab0093-4730-4ce3-b1f0-e8c0d6a9b80d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:30.496438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3ee90-9dfd-431a-b89d-e4d08a70d1dc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:30.500141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c381bf2-2161-4a8b-aa7c-b48ac48a2e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:30.500773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c381bf2-2161-4a8b-aa7c-b48ac48a2e99 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:30.501051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c381bf2-2161-4a8b-aa7c-b48ac48a2e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:30.501939 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c381bf2-2161-4a8b-aa7c-b48ac48a2e99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:30.513388 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c381bf2-2161-4a8b-aa7c-b48ac48a2e99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:30.514834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c381bf2-2161-4a8b-aa7c-b48ac48a2e99 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-4c381bf2-2161-4a8b-aa7c-b48ac48a2e99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:30.529600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:30.530554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:30.531438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:30.531438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:30.531438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:30.531438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 436/849] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:30.531438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c381bf2-2161-4a8b-aa7c-b48ac48a2e99 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:30.539527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baddbf-0c2f-42e8-a4ba-ca449c84c787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:30.540170 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baddbf-0c2f-42e8-a4ba-ca449c84c787 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:30.540458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baddbf-0c2f-42e8-a4ba-ca449c84c787 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:30.540715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baddbf-0c2f-42e8-a4ba-ca449c84c787 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:30.541685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75baddbf-0c2f-42e8-a4ba-ca449c84c787 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:30.541811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75baddbf-0c2f-42e8-a4ba-ca449c84c787 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:30.543130 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75baddbf-0c2f-42e8-a4ba-ca449c84c787 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:30.552573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 414/850] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:17:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:30.552573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baddbf-0c2f-42e8-a4ba-ca449c84c787 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:30.567798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:30.648327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c381bf2-2161-4a8b-aa7c-b48ac48a2e99 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:30.655395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baddbf-0c2f-42e8-a4ba-ca449c84c787 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:30.675981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:30.685377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257588b-2e2e-47a2-b6ce-060d30436098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:30.686021 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257588b-2e2e-47a2-b6ce-060d30436098 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:30.686321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257588b-2e2e-47a2-b6ce-060d30436098 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:30.686555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257588b-2e2e-47a2-b6ce-060d30436098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:30.693482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942ee42-2ffe-4e1e-b9af-323f1ff3f529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:30.694110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942ee42-2ffe-4e1e-b9af-323f1ff3f529 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:30.694389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942ee42-2ffe-4e1e-b9af-323f1ff3f529 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:30.694658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942ee42-2ffe-4e1e-b9af-323f1ff3f529 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:30.695325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942ee42-2ffe-4e1e-b9af-323f1ff3f529 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:30.695626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942ee42-2ffe-4e1e-b9af-323f1ff3f529 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:30.697086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942ee42-2ffe-4e1e-b9af-323f1ff3f529 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:30.700585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257588b-2e2e-47a2-b6ce-060d30436098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:30.702333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257588b-2e2e-47a2-b6ce-060d30436098 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:30.710259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 415/851] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:30.711462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942ee42-2ffe-4e1e-b9af-323f1ff3f529 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-b257588b-2e2e-47a2-b6ce-060d30436098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:30.721803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:30.722825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:30.723727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:30.723727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:30.723727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:30.723727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 437/852] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:30.723727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257588b-2e2e-47a2-b6ce-060d30436098 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:30.818588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:30.822577 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942ee42-2ffe-4e1e-b9af-323f1ff3f529 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:30.840470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257588b-2e2e-47a2-b6ce-060d30436098 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:30.857121 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b469b-5bd5-4aaa-8ed7-4ff33e158864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:30.857813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b469b-5bd5-4aaa-8ed7-4ff33e158864 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:30.858091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b469b-5bd5-4aaa-8ed7-4ff33e158864 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:30.858357 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b469b-5bd5-4aaa-8ed7-4ff33e158864 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:30.858973 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b469b-5bd5-4aaa-8ed7-4ff33e158864 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:30.859330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b469b-5bd5-4aaa-8ed7-4ff33e158864 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:30.861217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b469b-5bd5-4aaa-8ed7-4ff33e158864 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:30.865565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 416/853] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:17:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:30.866606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b469b-5bd5-4aaa-8ed7-4ff33e158864 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:30.873563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f79d60-0e7c-4846-9c6d-ab1737f4eaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:30.874321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f79d60-0e7c-4846-9c6d-ab1737f4eaa3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:30.874694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f79d60-0e7c-4846-9c6d-ab1737f4eaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:30.875130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f79d60-0e7c-4846-9c6d-ab1737f4eaa3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:30.884862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f79d60-0e7c-4846-9c6d-ab1737f4eaa3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:30.886402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f79d60-0e7c-4846-9c6d-ab1737f4eaa3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-93f79d60-0e7c-4846-9c6d-ab1737f4eaa3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:30.899953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:30.900999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:30.901950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:30.901950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:30.901950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:30.901950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:30.901950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:30.901950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:30.901950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:30.901950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:30.901950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 05:17:30.901950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:30.901950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 438/854] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 05:17:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:30.903699 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee1c18-3213-4567-a5dd-584e7758f442 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:30.903699 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee1c18-3213-4567-a5dd-584e7758f442 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:30.903699 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee1c18-3213-4567-a5dd-584e7758f442 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:30.904210 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 439/855] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:30.909564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee1c18-3213-4567-a5dd-584e7758f442 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:30.934010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:31.003875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b469b-5bd5-4aaa-8ed7-4ff33e158864 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:31.028823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee1c18-3213-4567-a5dd-584e7758f442 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:31.044821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad41751-4b62-4bfb-b711-b64d07e8dcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:31.045496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad41751-4b62-4bfb-b711-b64d07e8dcfc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:31.045861 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad41751-4b62-4bfb-b711-b64d07e8dcfc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:31.046125 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad41751-4b62-4bfb-b711-b64d07e8dcfc None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:31.046689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad41751-4b62-4bfb-b711-b64d07e8dcfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:31.046993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad41751-4b62-4bfb-b711-b64d07e8dcfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:31.048965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad41751-4b62-4bfb-b711-b64d07e8dcfc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:31.062662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 417/856] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:31.063738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8116464-5fac-4bfc-9aff-67e8fdfdcbfb None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:31.064225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8116464-5fac-4bfc-9aff-67e8fdfdcbfb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:31.064610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8116464-5fac-4bfc-9aff-67e8fdfdcbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:31.069175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325830e6-3d77-40ca-b1dd-f48b6af921bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:31.069473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325830e6-3d77-40ca-b1dd-f48b6af921bd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:31.069752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325830e6-3d77-40ca-b1dd-f48b6af921bd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:31.070060 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325830e6-3d77-40ca-b1dd-f48b6af921bd None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:31.070744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325830e6-3d77-40ca-b1dd-f48b6af921bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:31.071242 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325830e6-3d77-40ca-b1dd-f48b6af921bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:31.072712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325830e6-3d77-40ca-b1dd-f48b6af921bd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:31.081763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:31.083838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d8116464-5fac-4bfc-9aff-67e8fdfdcbfb None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:31.085082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d8116464-5fac-4bfc-9aff-67e8fdfdcbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:31.095339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 440/857] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 11 05:17:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:31.096220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325830e6-3d77-40ca-b1dd-f48b6af921bd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:31.160505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 418/858] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:31.162143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8116464-5fac-4bfc-9aff-67e8fdfdcbfb None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:31.212464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325830e6-3d77-40ca-b1dd-f48b6af921bd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:31.216211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:31.257760 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e710c27-dd51-4f68-9c28-7a33285ab0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:31.258392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e710c27-dd51-4f68-9c28-7a33285ab0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:31.258666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e710c27-dd51-4f68-9c28-7a33285ab0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:31.258935 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e710c27-dd51-4f68-9c28-7a33285ab0a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:31.265802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e710c27-dd51-4f68-9c28-7a33285ab0a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:31.267515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e710c27-dd51-4f68-9c28-7a33285ab0a9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-4e710c27-dd51-4f68-9c28-7a33285ab0a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:31.279150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:31.280338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:31.281238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:31.281238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:31.281238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:31.281238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:31.281238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:31.281238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:31.281238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:31.281238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:31.281238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 441/859] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:31.286578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e710c27-dd51-4f68-9c28-7a33285ab0a9 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:31.306977 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8116464-5fac-4bfc-9aff-67e8fdfdcbfb None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:31.354815 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:31.374064 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ee91e-d30b-46d6-b817-e56a4fd2f7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:31.374676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ee91e-d30b-46d6-b817-e56a4fd2f7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:31.374950 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ee91e-d30b-46d6-b817-e56a4fd2f7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:31.375214 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ee91e-d30b-46d6-b817-e56a4fd2f7e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:31.412148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e710c27-dd51-4f68-9c28-7a33285ab0a9 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:31.451477 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180e28f-136d-4011-bdb2-ba20f8e53a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:31.452493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180e28f-136d-4011-bdb2-ba20f8e53a62 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:31.452772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180e28f-136d-4011-bdb2-ba20f8e53a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:31.453052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180e28f-136d-4011-bdb2-ba20f8e53a62 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:31.453757 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180e28f-136d-4011-bdb2-ba20f8e53a62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:31.454091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180e28f-136d-4011-bdb2-ba20f8e53a62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:31.456276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180e28f-136d-4011-bdb2-ba20f8e53a62 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:31.466234 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ee91e-d30b-46d6-b817-e56a4fd2f7e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:31.467464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ee91e-d30b-46d6-b817-e56a4fd2f7e0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:31.474704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 442/860] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:31.474704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180e28f-136d-4011-bdb2-ba20f8e53a62 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:31.503900 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:31.577992 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 419/861] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:31 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:17:31.578753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ee91e-d30b-46d6-b817-e56a4fd2f7e0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:31.587849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180e28f-136d-4011-bdb2-ba20f8e53a62 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:31.606537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:31.626601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c791df-2b4e-446d-8680-810990e78003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:31.627322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c791df-2b4e-446d-8680-810990e78003 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:31.627605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c791df-2b4e-446d-8680-810990e78003 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:31.627871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c791df-2b4e-446d-8680-810990e78003 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:31.628485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c791df-2b4e-446d-8680-810990e78003 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:31.630115 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c791df-2b4e-446d-8680-810990e78003 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e5c791df-2b4e-446d-8680-810990e78003 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:31.638919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:31.640153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:31.641107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 443/862] 10.222.0.33 () {56 vars in 1132 bytes} [Sun Sep 11 05:17:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:31.642745 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c791df-2b4e-446d-8680-810990e78003 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:31.689869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ee91e-d30b-46d6-b817-e56a4fd2f7e0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:31.725137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5b83b-0f43-43a2-912c-d1e833aa67ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:31.725743 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5b83b-0f43-43a2-912c-d1e833aa67ca None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:31.726020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5b83b-0f43-43a2-912c-d1e833aa67ca None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:31.726286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5b83b-0f43-43a2-912c-d1e833aa67ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:31.727007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5b83b-0f43-43a2-912c-d1e833aa67ca None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=0a726949e8cc4cfdb25aa78f2761cff5) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:31.727310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5b83b-0f43-43a2-912c-d1e833aa67ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:31.728777 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5b83b-0f43-43a2-912c-d1e833aa67ca None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:31.730944 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:31.740170 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 420/863] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:17:31 2022] GET /identity/v3/role_assignments?user.id=0a726949e8cc4cfdb25aa78f2761cff5&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:31.742025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5b83b-0f43-43a2-912c-d1e833aa67ca None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:31.759593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c791df-2b4e-446d-8680-810990e78003 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:31.817514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771524f-d2ce-4314-9f08-c57e9f9eb276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:31.818011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771524f-d2ce-4314-9f08-c57e9f9eb276 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:31.818291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771524f-d2ce-4314-9f08-c57e9f9eb276 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:31.818564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771524f-d2ce-4314-9f08-c57e9f9eb276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:31.831563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771524f-d2ce-4314-9f08-c57e9f9eb276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:31.833107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771524f-d2ce-4314-9f08-c57e9f9eb276 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-3771524f-d2ce-4314-9f08-c57e9f9eb276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:31.850015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:31.851078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:31.852028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:31.852028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:31.852028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:31.856035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 444/864] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:31.857630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771524f-d2ce-4314-9f08-c57e9f9eb276 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:31.889635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5b83b-0f43-43a2-912c-d1e833aa67ca None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:31.894108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:31.931438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54bf34-72d8-4649-8b63-5f1cda78ecab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:31.932366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54bf34-72d8-4649-8b63-5f1cda78ecab None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:31.932645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54bf34-72d8-4649-8b63-5f1cda78ecab None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:31.933314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54bf34-72d8-4649-8b63-5f1cda78ecab None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:31.933676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54bf34-72d8-4649-8b63-5f1cda78ecab None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:31.933837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54bf34-72d8-4649-8b63-5f1cda78ecab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:31.935206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54bf34-72d8-4649-8b63-5f1cda78ecab None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:31.940659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 421/865] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 05:17:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:31.946237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54bf34-72d8-4649-8b63-5f1cda78ecab None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:31.972772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771524f-d2ce-4314-9f08-c57e9f9eb276 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:32.024170 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:32.026124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02503cd0-51b4-4ccf-bb60-57e4fae4b0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:32.026793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02503cd0-51b4-4ccf-bb60-57e4fae4b0df None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:32.027079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02503cd0-51b4-4ccf-bb60-57e4fae4b0df None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:32.027354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02503cd0-51b4-4ccf-bb60-57e4fae4b0df None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:32.028021 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02503cd0-51b4-4ccf-bb60-57e4fae4b0df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:32.028346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02503cd0-51b4-4ccf-bb60-57e4fae4b0df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:32.029751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02503cd0-51b4-4ccf-bb60-57e4fae4b0df None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:32.045544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 445/866] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:32.051146 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02503cd0-51b4-4ccf-bb60-57e4fae4b0df None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:32.075910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54bf34-72d8-4649-8b63-5f1cda78ecab None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:32.114812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d31bd2-a812-4ab5-a596-efc3a43e9e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:32.115726 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d31bd2-a812-4ab5-a596-efc3a43e9e4d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:32.116239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d31bd2-a812-4ab5-a596-efc3a43e9e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:32.116802 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d31bd2-a812-4ab5-a596-efc3a43e9e4d None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:32.117716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d31bd2-a812-4ab5-a596-efc3a43e9e4d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:32.118213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d31bd2-a812-4ab5-a596-efc3a43e9e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:32.119865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d31bd2-a812-4ab5-a596-efc3a43e9e4d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:32.125576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 422/867] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 05:17:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:32.156394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:32.165969 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02503cd0-51b4-4ccf-bb60-57e4fae4b0df None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:32.205308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7aa6b-3523-4317-b40d-d5cdb31d643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:32.205972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7aa6b-3523-4317-b40d-d5cdb31d643f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:32.206254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7aa6b-3523-4317-b40d-d5cdb31d643f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:32.206529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7aa6b-3523-4317-b40d-d5cdb31d643f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:32.214462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7aa6b-3523-4317-b40d-d5cdb31d643f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:32.215937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7aa6b-3523-4317-b40d-d5cdb31d643f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-bee7aa6b-3523-4317-b40d-d5cdb31d643f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:32.230095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:32.231274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:32.232342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:32.232342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:32.232342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:32.232342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:32.232342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:32.232342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:32.232342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:32.232342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:32.232342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 446/868] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:32.237432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d31bd2-a812-4ab5-a596-efc3a43e9e4d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:32.353677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d31bd2-a812-4ab5-a596-efc3a43e9e4d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:32.381641 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:32.391061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4fa29-4d0d-46c7-b112-f9fb10586e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:32.392410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4fa29-4d0d-46c7-b112-f9fb10586e53 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:32.392714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4fa29-4d0d-46c7-b112-f9fb10586e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:32.392882 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4fa29-4d0d-46c7-b112-f9fb10586e53 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:32.393482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4fa29-4d0d-46c7-b112-f9fb10586e53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:32.393791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4fa29-4d0d-46c7-b112-f9fb10586e53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:32.395233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4fa29-4d0d-46c7-b112-f9fb10586e53 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:32.408507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 423/869] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:32.414381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7aa6b-3523-4317-b40d-d5cdb31d643f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:32.519408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7aa6b-3523-4317-b40d-d5cdb31d643f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:32.554622 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3750c-e2d9-4935-a6af-3b98fffa0604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:32.555239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3750c-e2d9-4935-a6af-3b98fffa0604 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:32.555532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3750c-e2d9-4935-a6af-3b98fffa0604 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:32.555806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3750c-e2d9-4935-a6af-3b98fffa0604 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:32.556568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3750c-e2d9-4935-a6af-3b98fffa0604 None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=afd0a6d3b2194604a46767cfde12097b) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:32.556874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3750c-e2d9-4935-a6af-3b98fffa0604 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:32.558314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3750c-e2d9-4935-a6af-3b98fffa0604 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:32.565328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 447/870] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:17:32 2022] GET /identity/v3/role_assignments?user.id=afd0a6d3b2194604a46767cfde12097b&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:33.094240 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf14506-a6d1-40af-9e6e-70eb5a5d02f5 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:33.094665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf14506-a6d1-40af-9e6e-70eb5a5d02f5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:33.094863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf14506-a6d1-40af-9e6e-70eb5a5d02f5 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:33.096791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 424/871] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:33.102346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0577e01-c0ff-461f-b906-de51526aa41c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:33.102637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0577e01-c0ff-461f-b906-de51526aa41c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:33.102912 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0577e01-c0ff-461f-b906-de51526aa41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:33.121748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b0577e01-c0ff-461f-b906-de51526aa41c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:33.122943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b0577e01-c0ff-461f-b906-de51526aa41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:33.135730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:33.195041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 448/872] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:33.466543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf14506-a6d1-40af-9e6e-70eb5a5d02f5 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:33.575797 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf14506-a6d1-40af-9e6e-70eb5a5d02f5 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:33.611660 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d818b-1059-4438-900f-4c2eb42c4a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:33.612306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d818b-1059-4438-900f-4c2eb42c4a45 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:33.612586 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d818b-1059-4438-900f-4c2eb42c4a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:33.612854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d818b-1059-4438-900f-4c2eb42c4a45 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:33.613527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d818b-1059-4438-900f-4c2eb42c4a45 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:33.614877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d818b-1059-4438-900f-4c2eb42c4a45 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:33.635529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 425/873] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:17:33 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:17:34.095433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65333c80-dd3f-45b4-9baf-5285e66f3590 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:34.096476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65333c80-dd3f-45b4-9baf-5285e66f3590 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:34.096990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65333c80-dd3f-45b4-9baf-5285e66f3590 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:34.098767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 449/874] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:34.104265 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7c57d-0ff2-44d0-b7d8-dab7f92ae5af None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:34.104822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7c57d-0ff2-44d0-b7d8-dab7f92ae5af None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:34.105314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7c57d-0ff2-44d0-b7d8-dab7f92ae5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:34.123534 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d7a7c57d-0ff2-44d0-b7d8-dab7f92ae5af None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:34.125078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d7a7c57d-0ff2-44d0-b7d8-dab7f92ae5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:34.169921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:34.203119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 426/875] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:34 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:34.463226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34926dca-c5e5-40be-9284-f4c74db37cf2 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:34.463419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34926dca-c5e5-40be-9284-f4c74db37cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:34.463723 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34926dca-c5e5-40be-9284-f4c74db37cf2 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:34.465337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 450/876] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:34.471119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fb4a9-cf8b-4760-80ef-de805f2ddfa2 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:34.471580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fb4a9-cf8b-4760-80ef-de805f2ddfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:34.471674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fb4a9-cf8b-4760-80ef-de805f2ddfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:34.481246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34926dca-c5e5-40be-9284-f4c74db37cf2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:34.491431 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ca9fb4a9-cf8b-4760-80ef-de805f2ddfa2 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:34.492636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ca9fb4a9-cf8b-4760-80ef-de805f2ddfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:34.569862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 427/877] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:34 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:34.603200 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34926dca-c5e5-40be-9284-f4c74db37cf2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:34.637440 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965029fa-ad82-4f9c-9fce-e10c703bc0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:34.638365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965029fa-ad82-4f9c-9fce-e10c703bc0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:34.638847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965029fa-ad82-4f9c-9fce-e10c703bc0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:34.639318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965029fa-ad82-4f9c-9fce-e10c703bc0c8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:34.640147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965029fa-ad82-4f9c-9fce-e10c703bc0c8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:34.642488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965029fa-ad82-4f9c-9fce-e10c703bc0c8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-965029fa-ad82-4f9c-9fce-e10c703bc0c8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:34.651475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:34.652494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:34.653365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:34.656070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 451/878] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 05:17:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:34.662304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fb4a9-cf8b-4760-80ef-de805f2ddfa2 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:34.771315 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fb4a9-cf8b-4760-80ef-de805f2ddfa2 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:34.773187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:34.824229 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bfbbf-27d9-453c-8eb5-000352ef68f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:34.825225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bfbbf-27d9-453c-8eb5-000352ef68f0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:34.825712 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bfbbf-27d9-453c-8eb5-000352ef68f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:34.826205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bfbbf-27d9-453c-8eb5-000352ef68f0 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:34.827034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bfbbf-27d9-453c-8eb5-000352ef68f0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:34.827546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bfbbf-27d9-453c-8eb5-000352ef68f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:34.829227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bfbbf-27d9-453c-8eb5-000352ef68f0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:34.834465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 428/879] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 05:17:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:34.840258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965029fa-ad82-4f9c-9fce-e10c703bc0c8 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:34.850027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f52ae2-aeef-41b8-bc5e-9ff02016205a None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:34.850320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f52ae2-aeef-41b8-bc5e-9ff02016205a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:34.850602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f52ae2-aeef-41b8-bc5e-9ff02016205a None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:34.852178 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 429/880] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:34.855236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:34.857232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af9a3c-d4af-490f-822d-e59b956f7590 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:34.857518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af9a3c-d4af-490f-822d-e59b956f7590 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:34.859510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af9a3c-d4af-490f-822d-e59b956f7590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:34.879384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-68af9a3c-d4af-490f-822d-e59b956f7590 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:34.880607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-68af9a3c-d4af-490f-822d-e59b956f7590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:34.960043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 430/881] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:34 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:34.960238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965029fa-ad82-4f9c-9fce-e10c703bc0c8 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:34.960343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af9a3c-d4af-490f-822d-e59b956f7590 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:34.996043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480c210-f66d-4cb3-99ea-7624daf02c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:34.996662 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480c210-f66d-4cb3-99ea-7624daf02c7e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:34.996943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480c210-f66d-4cb3-99ea-7624daf02c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:34.997216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480c210-f66d-4cb3-99ea-7624daf02c7e None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:34.997853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480c210-f66d-4cb3-99ea-7624daf02c7e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:34.998261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480c210-f66d-4cb3-99ea-7624daf02c7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:34.999838 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480c210-f66d-4cb3-99ea-7624daf02c7e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:35.005370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 452/882] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 05:17:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:35.075389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af9a3c-d4af-490f-822d-e59b956f7590 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:35.077335 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:35.115849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eabb14-3b09-4008-a22d-c61d855e4ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:35.116782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eabb14-3b09-4008-a22d-c61d855e4ed9 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:35.117229 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eabb14-3b09-4008-a22d-c61d855e4ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:35.117664 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eabb14-3b09-4008-a22d-c61d855e4ed9 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:35.118478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eabb14-3b09-4008-a22d-c61d855e4ed9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:35.120345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eabb14-3b09-4008-a22d-c61d855e4ed9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:35.154695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 431/883] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:17:34 2022] POST /identity/v3/services => generated 269 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:17:35.264520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480c210-f66d-4cb3-99ea-7624daf02c7e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:35.357958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:35.388743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480c210-f66d-4cb3-99ea-7624daf02c7e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:35.430738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22431e09-3904-48ce-a267-e1afbc271c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:35.431374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22431e09-3904-48ce-a267-e1afbc271c47 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:35.431651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22431e09-3904-48ce-a267-e1afbc271c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:35.431922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22431e09-3904-48ce-a267-e1afbc271c47 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:35.445601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22431e09-3904-48ce-a267-e1afbc271c47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-22431e09-3904-48ce-a267-e1afbc271c47 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:35.450959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:35.451901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:35.452786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:35.452786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:35.452786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:35.452786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:35.452786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:35.452786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:35.452786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:35.452786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:35.452786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:35.452786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:35.452786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 05:17:35.452786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:35.452786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 453/884] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 11 05:17:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:35.458744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eabb14-3b09-4008-a22d-c61d855e4ed9 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:35.567402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eabb14-3b09-4008-a22d-c61d855e4ed9 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:35.603675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59f369-cb0c-4ad3-b7d7-30430e733c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:35.604643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59f369-cb0c-4ad3-b7d7-30430e733c4d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:35.605117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59f369-cb0c-4ad3-b7d7-30430e733c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:35.605563 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59f369-cb0c-4ad3-b7d7-30430e733c4d None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:35.606414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59f369-cb0c-4ad3-b7d7-30430e733c4d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:35.606908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59f369-cb0c-4ad3-b7d7-30430e733c4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:35.608559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59f369-cb0c-4ad3-b7d7-30430e733c4d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:35.622222 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 432/885] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 05:17:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:35.629133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22431e09-3904-48ce-a267-e1afbc271c47 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:35.741524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22431e09-3904-48ce-a267-e1afbc271c47 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:35.774489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:35.783570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30897b-41ec-4fa4-bd40-d2b37ec4c286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:35.784623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30897b-41ec-4fa4-bd40-d2b37ec4c286 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:35.785113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30897b-41ec-4fa4-bd40-d2b37ec4c286 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:35.785568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30897b-41ec-4fa4-bd40-d2b37ec4c286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:35.798568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30897b-41ec-4fa4-bd40-d2b37ec4c286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:35.800513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30897b-41ec-4fa4-bd40-d2b37ec4c286 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-9f30897b-41ec-4fa4-bd40-d2b37ec4c286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:35.816163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:35.817214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:35.818096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:35.818096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:35.818096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:35.818096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 454/886] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:35.823127 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59f369-cb0c-4ad3-b7d7-30430e733c4d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:35.846487 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:35.886887 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0373a7-5d59-426a-bed9-4ae7faf8059d None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:35.887078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0373a7-5d59-426a-bed9-4ae7faf8059d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:35.887340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0373a7-5d59-426a-bed9-4ae7faf8059d None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:35.888680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 455/887] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:35.894614 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07c6be-d63b-42fc-b84a-59ae491a71ca None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:35.894892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07c6be-d63b-42fc-b84a-59ae491a71ca None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:35.895162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07c6be-d63b-42fc-b84a-59ae491a71ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:35.913927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6f07c6be-d63b-42fc-b84a-59ae491a71ca None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:35.915211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6f07c6be-d63b-42fc-b84a-59ae491a71ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:35.936359 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59f369-cb0c-4ad3-b7d7-30430e733c4d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:35.980799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e22d7-bdb1-4fff-a1c5-d61bbf4bbad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:35.981736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e22d7-bdb1-4fff-a1c5-d61bbf4bbad3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:35.982025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e22d7-bdb1-4fff-a1c5-d61bbf4bbad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:35.982568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e22d7-bdb1-4fff-a1c5-d61bbf4bbad3 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:35.983219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e22d7-bdb1-4fff-a1c5-d61bbf4bbad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:35.983677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e22d7-bdb1-4fff-a1c5-d61bbf4bbad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:35.986506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e22d7-bdb1-4fff-a1c5-d61bbf4bbad3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:35.993142 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 456/888] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:36.003520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 433/889] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:36.009059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07c6be-d63b-42fc-b84a-59ae491a71ca None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:36.109599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07c6be-d63b-42fc-b84a-59ae491a71ca None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:36.123359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:36.145870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f7739-7bae-4f94-8629-0f6bfb1ff355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:36.146423 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f7739-7bae-4f94-8629-0f6bfb1ff355 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:36.146698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f7739-7bae-4f94-8629-0f6bfb1ff355 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:36.146963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f7739-7bae-4f94-8629-0f6bfb1ff355 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:36.154087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f7739-7bae-4f94-8629-0f6bfb1ff355 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:36.155523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f7739-7bae-4f94-8629-0f6bfb1ff355 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-146f7739-7bae-4f94-8629-0f6bfb1ff355 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:36.167619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:36.168829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:36.169862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:36.169862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:36.169862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:36.169862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:36.169862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:36.169862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:36.169862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:36.169862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:36.169862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 05:17:36.169862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:36.169862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 457/890] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 05:17:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:36.175340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e22d7-bdb1-4fff-a1c5-d61bbf4bbad3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:36.259816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:36.278235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f7739-7bae-4f94-8629-0f6bfb1ff355 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:36.287531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e22d7-bdb1-4fff-a1c5-d61bbf4bbad3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:36.335071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e028a-ebf4-498c-805b-91cecce85de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:36.336282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e028a-ebf4-498c-805b-91cecce85de0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:36.336763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e028a-ebf4-498c-805b-91cecce85de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:36.337699 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e028a-ebf4-498c-805b-91cecce85de0 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:36.338984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e028a-ebf4-498c-805b-91cecce85de0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:36.339503 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e028a-ebf4-498c-805b-91cecce85de0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:36.342048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e028a-ebf4-498c-805b-91cecce85de0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:36.372855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 434/891] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 11 05:17:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:36.375376 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e028a-ebf4-498c-805b-91cecce85de0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:36.405378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f7739-7bae-4f94-8629-0f6bfb1ff355 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:36.447113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dabf6e-0cee-4dcf-b4a3-56c5ed147805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:36.447748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dabf6e-0cee-4dcf-b4a3-56c5ed147805 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:36.448042 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dabf6e-0cee-4dcf-b4a3-56c5ed147805 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:36.448318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dabf6e-0cee-4dcf-b4a3-56c5ed147805 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:36.448898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dabf6e-0cee-4dcf-b4a3-56c5ed147805 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:36.450837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dabf6e-0cee-4dcf-b4a3-56c5ed147805 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-c8dabf6e-0cee-4dcf-b4a3-56c5ed147805 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:36.461593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:36.462618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 05:17:36.463459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:36.467762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 458/892] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:17:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:36.470761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dabf6e-0cee-4dcf-b4a3-56c5ed147805 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:36.514994 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e028a-ebf4-498c-805b-91cecce85de0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:36.517374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:36.566018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8d1ca-1760-4d50-8fb3-494404c583d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:36.567485 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8d1ca-1760-4d50-8fb3-494404c583d2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:36.568397 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8d1ca-1760-4d50-8fb3-494404c583d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:36.569320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8d1ca-1760-4d50-8fb3-494404c583d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:36.574685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:36.597715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dabf6e-0cee-4dcf-b4a3-56c5ed147805 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:36.642905 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda39b3-0e54-43b1-ac5d-490586a66b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:36.643840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda39b3-0e54-43b1-ac5d-490586a66b43 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:36.644318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda39b3-0e54-43b1-ac5d-490586a66b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:36.644768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda39b3-0e54-43b1-ac5d-490586a66b43 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:36.645598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda39b3-0e54-43b1-ac5d-490586a66b43 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:36.646484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda39b3-0e54-43b1-ac5d-490586a66b43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:36.648321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda39b3-0e54-43b1-ac5d-490586a66b43 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:36.654488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 459/893] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:17:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:36.661107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda39b3-0e54-43b1-ac5d-490586a66b43 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:36.666353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8d1ca-1760-4d50-8fb3-494404c583d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:36.667223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8d1ca-1760-4d50-8fb3-494404c583d2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:36.780414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 435/894] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:36 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 05:17:36.781841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8d1ca-1760-4d50-8fb3-494404c583d2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:36.796457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:36.803022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda39b3-0e54-43b1-ac5d-490586a66b43 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:36.840193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:36.842688 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58ffe7-d4bb-44bf-845b-1cea216a76e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:36.843331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58ffe7-d4bb-44bf-845b-1cea216a76e4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:36.843697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58ffe7-d4bb-44bf-845b-1cea216a76e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:36.843971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58ffe7-d4bb-44bf-845b-1cea216a76e4 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:36.845429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58ffe7-d4bb-44bf-845b-1cea216a76e4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:36.845744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58ffe7-d4bb-44bf-845b-1cea216a76e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:36.847096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58ffe7-d4bb-44bf-845b-1cea216a76e4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:36.857606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 460/895] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:17:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:36.859243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58ffe7-d4bb-44bf-845b-1cea216a76e4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:36.922354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8d1ca-1760-4d50-8fb3-494404c583d2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:36.961063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd4d44-7eee-4b9f-be9d-b52d3e3b85fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:36.961704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd4d44-7eee-4b9f-be9d-b52d3e3b85fb None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:36.962025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd4d44-7eee-4b9f-be9d-b52d3e3b85fb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:36.962300 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd4d44-7eee-4b9f-be9d-b52d3e3b85fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:36.974497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd4d44-7eee-4b9f-be9d-b52d3e3b85fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:36.975957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd4d44-7eee-4b9f-be9d-b52d3e3b85fb None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c1bd4d44-7eee-4b9f-be9d-b52d3e3b85fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:36.994444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:36.996320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:36.997306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:36.997306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:36.997306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:37.000207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 436/896] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:37.002782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd4d44-7eee-4b9f-be9d-b52d3e3b85fb None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:37.017085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58ffe7-d4bb-44bf-845b-1cea216a76e4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:37.064937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50083d71-9a13-4af7-a311-549e33300458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:37.066072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50083d71-9a13-4af7-a311-549e33300458 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:37.066354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50083d71-9a13-4af7-a311-549e33300458 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:37.066624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50083d71-9a13-4af7-a311-549e33300458 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:37.067476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50083d71-9a13-4af7-a311-549e33300458 None admin] RBAC: Adding query filter params (interface=public, service_id=d4e23f4253e644c686f473be8a433898, region_id=RegionOne) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:37.067917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50083d71-9a13-4af7-a311-549e33300458 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:37.069628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50083d71-9a13-4af7-a311-549e33300458 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:37.082579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 461/897] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 05:17:36 2022] GET /identity/v3/endpoints?service_id=d4e23f4253e644c686f473be8a433898&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:37.088184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50083d71-9a13-4af7-a311-549e33300458 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:37.134105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:37.140729 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:37.166747 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd4d44-7eee-4b9f-be9d-b52d3e3b85fb None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:37.226324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35591cf2-abb8-4a0e-9231-d781c3994e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:37.227431 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35591cf2-abb8-4a0e-9231-d781c3994e31 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:37.228216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35591cf2-abb8-4a0e-9231-d781c3994e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:37.229084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35591cf2-abb8-4a0e-9231-d781c3994e31 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:37.229974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35591cf2-abb8-4a0e-9231-d781c3994e31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:37.230486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35591cf2-abb8-4a0e-9231-d781c3994e31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:37.232473 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35591cf2-abb8-4a0e-9231-d781c3994e31 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:37.247744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50083d71-9a13-4af7-a311-549e33300458 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:37.254567 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 437/898] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:37.256819 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc5c22-021a-4b90-ae3c-9c70ee2afc02 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:37.257284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc5c22-021a-4b90-ae3c-9c70ee2afc02 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:37.257620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc5c22-021a-4b90-ae3c-9c70ee2afc02 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:37.259075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 438/899] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:37.259864 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc5c22-021a-4b90-ae3c-9c70ee2afc02 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:37.293746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cebcf-e067-45f6-8f0d-d3e25155e433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:37.295203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cebcf-e067-45f6-8f0d-d3e25155e433 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:37.295798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cebcf-e067-45f6-8f0d-d3e25155e433 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:37.296275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cebcf-e067-45f6-8f0d-d3e25155e433 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:37.296883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cebcf-e067-45f6-8f0d-d3e25155e433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:37.299150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cebcf-e067-45f6-8f0d-d3e25155e433 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:37.305872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 462/900] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 05:17:37 2022] GET /identity/v3/services => generated 1268 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:17:37.311154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90006f-c0e1-45f1-a3bd-4da697c84b87 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:37.311764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90006f-c0e1-45f1-a3bd-4da697c84b87 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:37.312344 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90006f-c0e1-45f1-a3bd-4da697c84b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:37.339898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2f90006f-c0e1-45f1-a3bd-4da697c84b87 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:37.342346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2f90006f-c0e1-45f1-a3bd-4da697c84b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:37.402564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc5c22-021a-4b90-ae3c-9c70ee2afc02 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:37.452035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 463/901] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:37.457763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47390407-ed48-44b9-bbed-321306acfae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:37.458481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47390407-ed48-44b9-bbed-321306acfae8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:37.458762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47390407-ed48-44b9-bbed-321306acfae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:37.459456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47390407-ed48-44b9-bbed-321306acfae8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:37.470820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47390407-ed48-44b9-bbed-321306acfae8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:37.472964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47390407-ed48-44b9-bbed-321306acfae8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-47390407-ed48-44b9-bbed-321306acfae8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:37.493657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:37.494596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:37.495507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:37.495507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:37.495507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:37.495507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:37.495507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:37.495507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:37.495507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:37.495507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:37.495507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 439/902] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:37.504039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90006f-c0e1-45f1-a3bd-4da697c84b87 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:37.530573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:37.566400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81b817-c2a5-4778-b1c9-cae7ca0867aa None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:37.566719 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81b817-c2a5-4778-b1c9-cae7ca0867aa None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:37.566948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81b817-c2a5-4778-b1c9-cae7ca0867aa None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:37.570620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 440/903] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:37.576606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a37cc-eab5-44b8-8aa4-2f97e0566937 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:37.577016 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a37cc-eab5-44b8-8aa4-2f97e0566937 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:37.577538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a37cc-eab5-44b8-8aa4-2f97e0566937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:37.601995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4c2a37cc-eab5-44b8-8aa4-2f97e0566937 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:37.603961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4c2a37cc-eab5-44b8-8aa4-2f97e0566937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:37.607211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:37.628044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90006f-c0e1-45f1-a3bd-4da697c84b87 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:37.671663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e01c66-07cc-4b68-b044-f941ca3cbc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:37.673570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e01c66-07cc-4b68-b044-f941ca3cbc89 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:37.674119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e01c66-07cc-4b68-b044-f941ca3cbc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:37.674464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e01c66-07cc-4b68-b044-f941ca3cbc89 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:37.675643 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e01c66-07cc-4b68-b044-f941ca3cbc89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:37.676199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e01c66-07cc-4b68-b044-f941ca3cbc89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:37.678654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e01c66-07cc-4b68-b044-f941ca3cbc89 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:37.693488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 464/904] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:37.699190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e01c66-07cc-4b68-b044-f941ca3cbc89 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:37.703516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 441/905] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:37.754522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a37cc-eab5-44b8-8aa4-2f97e0566937 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:37.803104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:37.825266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e01c66-07cc-4b68-b044-f941ca3cbc89 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:37.867778 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b41ec-fab6-4ab5-ace8-4e928a8334e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:37.868470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b41ec-fab6-4ab5-ace8-4e928a8334e4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:37.868750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b41ec-fab6-4ab5-ace8-4e928a8334e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:37.869020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b41ec-fab6-4ab5-ace8-4e928a8334e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:37.869810 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b41ec-fab6-4ab5-ace8-4e928a8334e4 None admin] RBAC: Adding query filter params (role.id=a1192a1e2adb45d490e8d553aa87c6a6, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=afd0a6d3b2194604a46767cfde12097b) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:37.870116 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b41ec-fab6-4ab5-ace8-4e928a8334e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:37.871544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b41ec-fab6-4ab5-ace8-4e928a8334e4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:37.878499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 465/906] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:17:37 2022] GET /identity/v3/role_assignments?user.id=afd0a6d3b2194604a46767cfde12097b&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=a1192a1e2adb45d490e8d553aa87c6a6 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:37.908940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a37cc-eab5-44b8-8aa4-2f97e0566937 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:37.950420 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724976ea-b8db-4453-a3fd-b20e51e1734d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:37.951127 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724976ea-b8db-4453-a3fd-b20e51e1734d None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:37.951413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724976ea-b8db-4453-a3fd-b20e51e1734d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:37.951686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724976ea-b8db-4453-a3fd-b20e51e1734d None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:37.952517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724976ea-b8db-4453-a3fd-b20e51e1734d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:37.954020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724976ea-b8db-4453-a3fd-b20e51e1734d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:37.964964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:37.976502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 442/907] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:17:37 2022] POST /identity/v3/services => generated 241 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:17:38.014770 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b41ec-fab6-4ab5-ace8-4e928a8334e4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:38.142553 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b41ec-fab6-4ab5-ace8-4e928a8334e4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:38.188283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba5382-0b63-48f0-904f-9d88b2e06e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:38.189381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba5382-0b63-48f0-904f-9d88b2e06e9f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:38.189873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba5382-0b63-48f0-904f-9d88b2e06e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:38.190354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba5382-0b63-48f0-904f-9d88b2e06e9f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:38.191139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba5382-0b63-48f0-904f-9d88b2e06e9f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:38.192670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba5382-0b63-48f0-904f-9d88b2e06e9f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-29ba5382-0b63-48f0-904f-9d88b2e06e9f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:38.202209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:38.203389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 05:17:38.205093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:38.206026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 466/908] 10.222.0.33 () {56 vars in 1132 bytes} [Sun Sep 11 05:17:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:38.213101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724976ea-b8db-4453-a3fd-b20e51e1734d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:38.323029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724976ea-b8db-4453-a3fd-b20e51e1734d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:38.363372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d160b2b-9e7d-4459-98f9-ae118b72a706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:38.363989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d160b2b-9e7d-4459-98f9-ae118b72a706 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:38.364373 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d160b2b-9e7d-4459-98f9-ae118b72a706 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:38.364651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d160b2b-9e7d-4459-98f9-ae118b72a706 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:38.365286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d160b2b-9e7d-4459-98f9-ae118b72a706 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:38.365592 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d160b2b-9e7d-4459-98f9-ae118b72a706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:38.366969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d160b2b-9e7d-4459-98f9-ae118b72a706 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:38.372299 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 443/909] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 05:17:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:38.378068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba5382-0b63-48f0-904f-9d88b2e06e9f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:38.400371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:38.484614 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba5382-0b63-48f0-904f-9d88b2e06e9f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:38.520156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab955027-d1ae-4e9a-91a1-d41850f1e855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:38.521012 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab955027-d1ae-4e9a-91a1-d41850f1e855 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:38.521435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab955027-d1ae-4e9a-91a1-d41850f1e855 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:38.521863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab955027-d1ae-4e9a-91a1-d41850f1e855 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:38.522627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab955027-d1ae-4e9a-91a1-d41850f1e855 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:38.523069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab955027-d1ae-4e9a-91a1-d41850f1e855 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:38.524604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab955027-d1ae-4e9a-91a1-d41850f1e855 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:38.535127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 467/910] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 05:17:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:38.536911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d160b2b-9e7d-4459-98f9-ae118b72a706 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:38.602611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:38.646388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d160b2b-9e7d-4459-98f9-ae118b72a706 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:38.681308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabbc51-fb66-4a34-826c-eb7c6b50c892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:38.681923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabbc51-fb66-4a34-826c-eb7c6b50c892 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:38.682202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabbc51-fb66-4a34-826c-eb7c6b50c892 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:38.682474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabbc51-fb66-4a34-826c-eb7c6b50c892 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:38.683187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabbc51-fb66-4a34-826c-eb7c6b50c892 None admin] RBAC: Adding query filter params (interface=public, service_id=aab751ead75041a28fe7713e6e12913e, region_id=RegionOne) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:38.683559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabbc51-fb66-4a34-826c-eb7c6b50c892 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:38.684963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabbc51-fb66-4a34-826c-eb7c6b50c892 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:38.691278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 444/911] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 05:17:38 2022] GET /identity/v3/endpoints?service_id=aab751ead75041a28fe7713e6e12913e&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:38.697919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab955027-d1ae-4e9a-91a1-d41850f1e855 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:38.771141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:38.810803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab955027-d1ae-4e9a-91a1-d41850f1e855 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:38.845898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21829185-e66b-4d14-bafa-cfb25c9c1a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:38.846502 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21829185-e66b-4d14-bafa-cfb25c9c1a77 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:38.846780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21829185-e66b-4d14-bafa-cfb25c9c1a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:38.847049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21829185-e66b-4d14-bafa-cfb25c9c1a77 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:38.847643 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21829185-e66b-4d14-bafa-cfb25c9c1a77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:38.849027 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21829185-e66b-4d14-bafa-cfb25c9c1a77 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:38.860744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 468/912] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 05:17:38 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:17:39.682206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf133587-6a97-4cb4-8402-1726396724eb None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:39.683245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf133587-6a97-4cb4-8402-1726396724eb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:39.683734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf133587-6a97-4cb4-8402-1726396724eb None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:39.685572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 445/913] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:39.691039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841bcb1-0fdd-4edd-9d8a-2b7cc624f91f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:39.691879 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841bcb1-0fdd-4edd-9d8a-2b7cc624f91f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:39.692394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841bcb1-0fdd-4edd-9d8a-2b7cc624f91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:39.711794 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c841bcb1-0fdd-4edd-9d8a-2b7cc624f91f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:39.713368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c841bcb1-0fdd-4edd-9d8a-2b7cc624f91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:39.784290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 469/914] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:40.064401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf133587-6a97-4cb4-8402-1726396724eb None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:40.073515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6026a0-b75a-4638-b4e0-f9820f2b4cd1 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:40.073821 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6026a0-b75a-4638-b4e0-f9820f2b4cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:40.074065 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6026a0-b75a-4638-b4e0-f9820f2b4cd1 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:40.075654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 470/915] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:40.080751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e8167-51ca-4212-b78d-e91af2fe0fea None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:40.081033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e8167-51ca-4212-b78d-e91af2fe0fea None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:40.081311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e8167-51ca-4212-b78d-e91af2fe0fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:40.100401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-892e8167-51ca-4212-b78d-e91af2fe0fea None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:40.101583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-892e8167-51ca-4212-b78d-e91af2fe0fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:40.184853 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf133587-6a97-4cb4-8402-1726396724eb None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:40.186738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:40.202375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 471/916] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:40.219306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:40.230262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d83d6e-a3bc-489e-873a-67bc5ae22b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:40.231175 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d83d6e-a3bc-489e-873a-67bc5ae22b25 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:40.231653 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d83d6e-a3bc-489e-873a-67bc5ae22b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:40.232208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d83d6e-a3bc-489e-873a-67bc5ae22b25 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:40.233041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d83d6e-a3bc-489e-873a-67bc5ae22b25 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:40.234604 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d83d6e-a3bc-489e-873a-67bc5ae22b25 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c4d83d6e-a3bc-489e-873a-67bc5ae22b25 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:40.244232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:40.245208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 05:17:40.246053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:40.248753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 446/917] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:17:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:40.257606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e8167-51ca-4212-b78d-e91af2fe0fea None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:40.291279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7859fd2-dd25-4ebd-94cd-ed3e004d1f4a None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:40.291906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7859fd2-dd25-4ebd-94cd-ed3e004d1f4a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:40.292417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7859fd2-dd25-4ebd-94cd-ed3e004d1f4a None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:40.294504 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 447/918] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:40.299602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d522b6-8e37-4fe8-8e21-e4b11767f779 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:40.299896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d522b6-8e37-4fe8-8e21-e4b11767f779 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:40.300209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d522b6-8e37-4fe8-8e21-e4b11767f779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:40.319370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f0d522b6-8e37-4fe8-8e21-e4b11767f779 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:40.320701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f0d522b6-8e37-4fe8-8e21-e4b11767f779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:40.369325 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e8167-51ca-4212-b78d-e91af2fe0fea None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:40.394697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 448/919] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:40.409422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7154a6e-5812-4cbf-9dee-190862e21930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:40.410229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7154a6e-5812-4cbf-9dee-190862e21930 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:40.410651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7154a6e-5812-4cbf-9dee-190862e21930 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:40.411227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7154a6e-5812-4cbf-9dee-190862e21930 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:40.412052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7154a6e-5812-4cbf-9dee-190862e21930 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:40.412495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7154a6e-5812-4cbf-9dee-190862e21930 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:40.413995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7154a6e-5812-4cbf-9dee-190862e21930 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:40.419144 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 472/920] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:17:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:40.425581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d522b6-8e37-4fe8-8e21-e4b11767f779 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:40.470754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7154a6e-5812-4cbf-9dee-190862e21930 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:40.543829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d522b6-8e37-4fe8-8e21-e4b11767f779 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:40.549769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:40.573244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:40.592586 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75759ad5-8552-4973-9ce3-3d27ae9ab2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:40.593235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75759ad5-8552-4973-9ce3-3d27ae9ab2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:40.593514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75759ad5-8552-4973-9ce3-3d27ae9ab2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:40.593803 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75759ad5-8552-4973-9ce3-3d27ae9ab2a7 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:40.595106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75759ad5-8552-4973-9ce3-3d27ae9ab2a7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:40.595426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75759ad5-8552-4973-9ce3-3d27ae9ab2a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:40.597202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75759ad5-8552-4973-9ce3-3d27ae9ab2a7 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:40.601869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7154a6e-5812-4cbf-9dee-190862e21930 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:40.607620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 449/921] 10.222.0.33 () {56 vars in 1114 bytes} [Sun Sep 11 05:17:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:40.608994 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75759ad5-8552-4973-9ce3-3d27ae9ab2a7 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:40.651381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22aeb8-a6d6-43a6-907f-6cad09b0cf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:40.652033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22aeb8-a6d6-43a6-907f-6cad09b0cf4f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:40.652318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22aeb8-a6d6-43a6-907f-6cad09b0cf4f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:40.652587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22aeb8-a6d6-43a6-907f-6cad09b0cf4f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:40.665554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22aeb8-a6d6-43a6-907f-6cad09b0cf4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-ad22aeb8-a6d6-43a6-907f-6cad09b0cf4f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:40.671293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:40.672288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:40.673706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:40.673706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:40.673706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:40.673706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:40.673706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:40.673706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:40.673706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:40.673706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:40.673706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:40.673706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:40.673706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 05:17:40.673706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:40.676022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 473/922] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 11 05:17:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:40.677037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22aeb8-a6d6-43a6-907f-6cad09b0cf4f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:40.725558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75759ad5-8552-4973-9ce3-3d27ae9ab2a7 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:40.762601 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94977ef9-df6c-4669-90b8-04589f6b7a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:40.763482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94977ef9-df6c-4669-90b8-04589f6b7a7c None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:40.763929 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94977ef9-df6c-4669-90b8-04589f6b7a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:40.764423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94977ef9-df6c-4669-90b8-04589f6b7a7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:40.765235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94977ef9-df6c-4669-90b8-04589f6b7a7c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:40.766934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94977ef9-df6c-4669-90b8-04589f6b7a7c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:40.783674 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22aeb8-a6d6-43a6-907f-6cad09b0cf4f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:40.804741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 450/923] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:17:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:17:40.804741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94977ef9-df6c-4669-90b8-04589f6b7a7c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:40.815887 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:40.853061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc73951-3878-4cba-85aa-5b9a1ff0ad66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:40.853741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc73951-3878-4cba-85aa-5b9a1ff0ad66 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:40.854097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc73951-3878-4cba-85aa-5b9a1ff0ad66 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:40.854436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc73951-3878-4cba-85aa-5b9a1ff0ad66 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:40.855096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc73951-3878-4cba-85aa-5b9a1ff0ad66 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:40.857139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc73951-3878-4cba-85aa-5b9a1ff0ad66 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:40.858799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-bfc73951-3878-4cba-85aa-5b9a1ff0ad66 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:40.874434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:40.879187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:40.880780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:40.884759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 474/924] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 05:17:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:40.884759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc73951-3878-4cba-85aa-5b9a1ff0ad66 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:40.939962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94977ef9-df6c-4669-90b8-04589f6b7a7c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:40.985598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4089f-7329-48e4-a936-3dbe905205d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:40.986803 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4089f-7329-48e4-a936-3dbe905205d3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:40.987277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4089f-7329-48e4-a936-3dbe905205d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:40.988384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4089f-7329-48e4-a936-3dbe905205d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:40.989067 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4089f-7329-48e4-a936-3dbe905205d3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:40.989376 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4089f-7329-48e4-a936-3dbe905205d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:40.995346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4089f-7329-48e4-a936-3dbe905205d3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:41.009103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 451/925] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 05:17:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:41.017409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4089f-7329-48e4-a936-3dbe905205d3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:41.039384 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc73951-3878-4cba-85aa-5b9a1ff0ad66 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:41.080186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17c1e2-89dd-462e-9c81-bf734704b008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:41.081259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17c1e2-89dd-462e-9c81-bf734704b008 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:41.081550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17c1e2-89dd-462e-9c81-bf734704b008 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:41.081823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17c1e2-89dd-462e-9c81-bf734704b008 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:41.084608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17c1e2-89dd-462e-9c81-bf734704b008 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:41.085125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17c1e2-89dd-462e-9c81-bf734704b008 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:41.087771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d17c1e2-89dd-462e-9c81-bf734704b008 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:41.094878 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 475/926] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 05:17:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:41.100244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17c1e2-89dd-462e-9c81-bf734704b008 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:41.135493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4089f-7329-48e4-a936-3dbe905205d3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:41.136995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:41.152438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:41.172569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb3a49-d3a3-440d-8f61-f440b8adfcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:41.173427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb3a49-d3a3-440d-8f61-f440b8adfcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:41.173804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb3a49-d3a3-440d-8f61-f440b8adfcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:41.174182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb3a49-d3a3-440d-8f61-f440b8adfcb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:41.186017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb3a49-d3a3-440d-8f61-f440b8adfcb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:41.187480 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb3a49-d3a3-440d-8f61-f440b8adfcb9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-d5cb3a49-d3a3-440d-8f61-f440b8adfcb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:41.201620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:41.207828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:41.208845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:41.208845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:41.208845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:41.208845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 452/927] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:41.208845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16714ab2-43ef-4436-a5df-df7abda6b30b None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:41.208845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16714ab2-43ef-4436-a5df-df7abda6b30b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:41.208845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16714ab2-43ef-4436-a5df-df7abda6b30b None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:41.208845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 453/928] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:41.209329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16714ab2-43ef-4436-a5df-df7abda6b30b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:41.223084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17c1e2-89dd-462e-9c81-bf734704b008 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:41.259121 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b1aa4-5157-4333-9153-647decc0c960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:41.259736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b1aa4-5157-4333-9153-647decc0c960 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:41.260028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b1aa4-5157-4333-9153-647decc0c960 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:41.260308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b1aa4-5157-4333-9153-647decc0c960 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:41.260942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b1aa4-5157-4333-9153-647decc0c960 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:41.261243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b1aa4-5157-4333-9153-647decc0c960 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:41.262604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b1aa4-5157-4333-9153-647decc0c960 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:41.272874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 476/929] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 05:17:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:41.272874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad28f17-3a89-4099-8205-ece634a5f110 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:41.272874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad28f17-3a89-4099-8205-ece634a5f110 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:41.272874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad28f17-3a89-4099-8205-ece634a5f110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:41.288461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2ad28f17-3a89-4099-8205-ece634a5f110 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:41.289645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2ad28f17-3a89-4099-8205-ece634a5f110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:41.322946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16714ab2-43ef-4436-a5df-df7abda6b30b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:41.362863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981e228-dff2-4719-8d9b-4c8e34eee25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:41.363477 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981e228-dff2-4719-8d9b-4c8e34eee25d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:41.363762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981e228-dff2-4719-8d9b-4c8e34eee25d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:41.364118 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981e228-dff2-4719-8d9b-4c8e34eee25d None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:41.364713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981e228-dff2-4719-8d9b-4c8e34eee25d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:41.365097 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981e228-dff2-4719-8d9b-4c8e34eee25d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:41.372345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 477/930] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:41.372544 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981e228-dff2-4719-8d9b-4c8e34eee25d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:41.372640 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad28f17-3a89-4099-8205-ece634a5f110 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:41.396243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 454/931] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:41.400231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981e228-dff2-4719-8d9b-4c8e34eee25d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:41.406966 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:41.421502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:41.480932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad28f17-3a89-4099-8205-ece634a5f110 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:41.527517 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63843b3a-8073-487d-b033-15ae436cf191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:41.528154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63843b3a-8073-487d-b033-15ae436cf191 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:41.528435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63843b3a-8073-487d-b033-15ae436cf191 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:41.528708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63843b3a-8073-487d-b033-15ae436cf191 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:41.529415 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63843b3a-8073-487d-b033-15ae436cf191 None admin] RBAC: Adding query filter params (interface=public, service_id=4996407a6ae94e0791fc4e0633a3cd40, region_id=RegionOne) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:41.529722 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63843b3a-8073-487d-b033-15ae436cf191 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:41.531772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63843b3a-8073-487d-b033-15ae436cf191 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:41.540040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 478/932] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 05:17:41 2022] GET /identity/v3/endpoints?service_id=4996407a6ae94e0791fc4e0633a3cd40&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:41.545045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63843b3a-8073-487d-b033-15ae436cf191 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:41.546279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981e228-dff2-4719-8d9b-4c8e34eee25d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:41.586811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e5366-ca6d-446c-a11c-baa48355a15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:41.587569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e5366-ca6d-446c-a11c-baa48355a15a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:41.587910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e5366-ca6d-446c-a11c-baa48355a15a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:41.588271 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e5366-ca6d-446c-a11c-baa48355a15a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:41.597107 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e5366-ca6d-446c-a11c-baa48355a15a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:41.598778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e5366-ca6d-446c-a11c-baa48355a15a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-922e5366-ca6d-446c-a11c-baa48355a15a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:41.612341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:41.613352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:41.614288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:41.614288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:41.614288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:41.614288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:41.614288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:41.614288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:41.614288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:41.614288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:41.614288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 05:17:41.614288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:41.614288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 455/933] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 05:17:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:41.621026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e5366-ca6d-446c-a11c-baa48355a15a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:41.667820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:41.671924 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63843b3a-8073-487d-b033-15ae436cf191 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:41.709520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64180bf4-05fb-4795-b8eb-33a6525054a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:41.710121 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64180bf4-05fb-4795-b8eb-33a6525054a7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:41.710398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64180bf4-05fb-4795-b8eb-33a6525054a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:41.710665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64180bf4-05fb-4795-b8eb-33a6525054a7 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:41.711268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64180bf4-05fb-4795-b8eb-33a6525054a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:41.712665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64180bf4-05fb-4795-b8eb-33a6525054a7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:41.713155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:41.728040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 479/934] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 05:17:41 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:17:41.728040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64180bf4-05fb-4795-b8eb-33a6525054a7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:41.755541 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e5366-ca6d-446c-a11c-baa48355a15a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:41.808247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5608cf-2d18-4454-b768-3994fc5b173f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:41.809426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5608cf-2d18-4454-b768-3994fc5b173f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:41.810420 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5608cf-2d18-4454-b768-3994fc5b173f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:41.810420 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5608cf-2d18-4454-b768-3994fc5b173f None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:41.811028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5608cf-2d18-4454-b768-3994fc5b173f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:41.811405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5608cf-2d18-4454-b768-3994fc5b173f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:41.813801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5608cf-2d18-4454-b768-3994fc5b173f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:41.839409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 456/935] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 11 05:17:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:41.845116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5608cf-2d18-4454-b768-3994fc5b173f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:41.855197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64180bf4-05fb-4795-b8eb-33a6525054a7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:41.894527 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e22cf6-31a3-4dca-b344-06c4316577e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:41.895187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e22cf6-31a3-4dca-b344-06c4316577e2 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:41.895472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e22cf6-31a3-4dca-b344-06c4316577e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:41.895749 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e22cf6-31a3-4dca-b344-06c4316577e2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:41.896354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e22cf6-31a3-4dca-b344-06c4316577e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:41.897758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e22cf6-31a3-4dca-b344-06c4316577e2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-19e22cf6-31a3-4dca-b344-06c4316577e2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:41.909816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:41.910871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:41.912151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 480/936] 10.222.0.33 () {56 vars in 1132 bytes} [Sun Sep 11 05:17:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:41.916872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e22cf6-31a3-4dca-b344-06c4316577e2 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:41.953646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:41.987119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5608cf-2d18-4454-b768-3994fc5b173f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:42.032855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:42.039355 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342b8eb-9776-4ab0-87f7-a0a0a67544cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:42.040154 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342b8eb-9776-4ab0-87f7-a0a0a67544cf None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:42.040539 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342b8eb-9776-4ab0-87f7-a0a0a67544cf None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:42.041382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342b8eb-9776-4ab0-87f7-a0a0a67544cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:42.055347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342b8eb-9776-4ab0-87f7-a0a0a67544cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:42.056145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e22cf6-31a3-4dca-b344-06c4316577e2 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:42.057071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342b8eb-9776-4ab0-87f7-a0a0a67544cf None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-d342b8eb-9776-4ab0-87f7-a0a0a67544cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:42.075741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:42.076922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:42.077828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:42.077828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:42.077828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:42.078277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 457/937] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:42.084081 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342b8eb-9776-4ab0-87f7-a0a0a67544cf None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:42.105009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8b424-24b2-4e08-8343-ca403b79fabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:42.106395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8b424-24b2-4e08-8343-ca403b79fabd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:42.106877 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8b424-24b2-4e08-8343-ca403b79fabd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:42.107376 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8b424-24b2-4e08-8343-ca403b79fabd None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:42.108230 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8b424-24b2-4e08-8343-ca403b79fabd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:42.109303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8b424-24b2-4e08-8343-ca403b79fabd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:42.111312 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8b424-24b2-4e08-8343-ca403b79fabd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:42.117529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 481/938] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 05:17:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:42.122860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8b424-24b2-4e08-8343-ca403b79fabd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:42.198055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342b8eb-9776-4ab0-87f7-a0a0a67544cf None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:42.236793 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbd5b6-8fb1-4dfa-bb17-b8b01b1071a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:42.237702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd5b6-8fb1-4dfa-bb17-b8b01b1071a8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:42.238253 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd5b6-8fb1-4dfa-bb17-b8b01b1071a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:42.238726 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd5b6-8fb1-4dfa-bb17-b8b01b1071a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:42.239555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbd5b6-8fb1-4dfa-bb17-b8b01b1071a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:42.240074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbd5b6-8fb1-4dfa-bb17-b8b01b1071a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:42.241877 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:42.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbd5b6-8fb1-4dfa-bb17-b8b01b1071a8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:42.253622 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8b424-24b2-4e08-8343-ca403b79fabd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:42.259904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 458/939] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:42.266947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbd5b6-8fb1-4dfa-bb17-b8b01b1071a8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:42.302495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0109c1-2aa4-450e-a905-43bd3480d5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:42.303410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0109c1-2aa4-450e-a905-43bd3480d5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:42.303951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0109c1-2aa4-450e-a905-43bd3480d5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:42.304451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0109c1-2aa4-450e-a905-43bd3480d5b3 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:42.305305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0109c1-2aa4-450e-a905-43bd3480d5b3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:42.305795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0109c1-2aa4-450e-a905-43bd3480d5b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:42.307392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0109c1-2aa4-450e-a905-43bd3480d5b3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:42.313147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 482/940] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 05:17:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:42.318119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:42.319213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0109c1-2aa4-450e-a905-43bd3480d5b3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:42.403049 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbd5b6-8fb1-4dfa-bb17-b8b01b1071a8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:42.434304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0109c1-2aa4-450e-a905-43bd3480d5b3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:42.452759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8d5be-7289-4dad-b0d2-5c507dee4a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:42.454161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8d5be-7289-4dad-b0d2-5c507dee4a2a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:42.454631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8d5be-7289-4dad-b0d2-5c507dee4a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:42.455085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8d5be-7289-4dad-b0d2-5c507dee4a2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:42.466218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8d5be-7289-4dad-b0d2-5c507dee4a2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:42.469033 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8d5be-7289-4dad-b0d2-5c507dee4a2a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:42.472995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f9607-8495-436a-8e19-0a8ff0198cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:42.473608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f9607-8495-436a-8e19-0a8ff0198cef None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:42.473893 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f9607-8495-436a-8e19-0a8ff0198cef None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:42.474163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f9607-8495-436a-8e19-0a8ff0198cef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:42.474831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f9607-8495-436a-8e19-0a8ff0198cef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:42.476223 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f9607-8495-436a-8e19-0a8ff0198cef None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-bbe8d5be-7289-4dad-b0d2-5c507dee4a2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:42.483953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:42.484865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:42.485912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:42.485912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:42.485912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:42.485912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:42.485912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:42.485912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:42.485912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:42.485912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:42.488715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 459/941] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:42.495417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8d5be-7289-4dad-b0d2-5c507dee4a2a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:42.504106 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 483/942] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:17:42 2022] POST /identity/v3/endpoints => generated 347 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:17:42.620037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8d5be-7289-4dad-b0d2-5c507dee4a2a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:42.623084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:42.669016 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc6256-7029-4604-ba6c-457e61487113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:42.669993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6256-7029-4604-ba6c-457e61487113 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:42.670463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6256-7029-4604-ba6c-457e61487113 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:42.670922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6256-7029-4604-ba6c-457e61487113 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:42.671881 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc6256-7029-4604-ba6c-457e61487113 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:42.672392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc6256-7029-4604-ba6c-457e61487113 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:42.674622 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc6256-7029-4604-ba6c-457e61487113 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:42.687792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 460/943] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:42.694088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f9607-8495-436a-8e19-0a8ff0198cef None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:42.713970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:42.809788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f9607-8495-436a-8e19-0a8ff0198cef None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:42.846676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9abc6-729f-4436-a7a7-40bba1b01068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:42.847572 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9abc6-729f-4436-a7a7-40bba1b01068 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:42.848014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9abc6-729f-4436-a7a7-40bba1b01068 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:42.848473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9abc6-729f-4436-a7a7-40bba1b01068 None admin] PATH_INFO: `/v3/projects/092dd30ff8b541e2a45947504831998a/users/afd0a6d3b2194604a46767cfde12097b/roles/a1192a1e2adb45d490e8d553aa87c6a6` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:42.850758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9abc6-729f-4436-a7a7-40bba1b01068 None admin] RBAC: Authorizing `identity:create_grant(project_id=092dd30ff8b541e2a45947504831998a, user_id=afd0a6d3b2194604a46767cfde12097b, role_id=a1192a1e2adb45d490e8d553aa87c6a6)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:42.852367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9abc6-729f-4436-a7a7-40bba1b01068 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:42.868762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 484/944] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:17:42 2022] PUT /identity/v3/projects/092dd30ff8b541e2a45947504831998a/users/afd0a6d3b2194604a46767cfde12097b/roles/a1192a1e2adb45d490e8d553aa87c6a6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:17:43.099405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c02c70-cac7-49d0-be62-2a6553a3274c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:43.099958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c02c70-cac7-49d0-be62-2a6553a3274c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:43.100083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c02c70-cac7-49d0-be62-2a6553a3274c None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:43.101505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 461/945] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:43.106900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c2888-bb91-4de8-89a7-8d4f44e0dc44 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:43.107363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c2888-bb91-4de8-89a7-8d4f44e0dc44 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:43.107805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c2888-bb91-4de8-89a7-8d4f44e0dc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:43.127477 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4a8c2888-bb91-4de8-89a7-8d4f44e0dc44 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:43.129265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4a8c2888-bb91-4de8-89a7-8d4f44e0dc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:43.197895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 485/946] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:43.471396 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c02c70-cac7-49d0-be62-2a6553a3274c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:43.583345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c02c70-cac7-49d0-be62-2a6553a3274c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:43.619616 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714529c8-d5ba-4078-a821-b04b801be750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:43.620531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714529c8-d5ba-4078-a821-b04b801be750 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:43.620960 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714529c8-d5ba-4078-a821-b04b801be750 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:43.621380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714529c8-d5ba-4078-a821-b04b801be750 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:43.622209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714529c8-d5ba-4078-a821-b04b801be750 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:43.623794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714529c8-d5ba-4078-a821-b04b801be750 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:43.627219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: WARNING keystone.server.flask.common [None req-714529c8-d5ba-4078-a821-b04b801be750 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 11 05:17:43.648649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.sql.core [None req-714529c8-d5ba-4078-a821-b04b801be750 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 05:17:43.648649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 11 05:17:43.648649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [parameters: {'id': '73482d710c504010a4476f2f455b7b89', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 11 05:17:43.648649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81445) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 05:17:43.656438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-714529c8-d5ba-4078-a821-b04b801be750 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 73482d710c504010a4476f2f455b7b89.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 73482d710c504010a4476f2f455b7b89. Sep 11 05:17:43.656438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:43.656438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 05:17:43.656438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 05:17:43.656438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 05:17:43.656438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 05:17:43.656438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 05:17:43.656438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 05:17:43.656438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 05:17:43.656438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application conn.query(q) Sep 11 05:17:43.656438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 05:17:43.656438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 05:17:43.656438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application result.read() Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 11 05:17:43.657564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application next(self.gen) Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application yield resource Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application next(self.gen) Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise self.value Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application session.commit() Sep 11 05:17:43.658771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.session.flush() Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application compat.raise_( Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 05:17:43.659914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 05:17:43.660886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application conn.query(q) Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application result.read() Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 05:17:43.661695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [parameters: {'id': '73482d710c504010a4476f2f455b7b89', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:43.662589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:43.663525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:43.664512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:43.664512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 11 05:17:43.664512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 11 05:17:43.664512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:43.664512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:43.664512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 11 05:17:43.664512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 11 05:17:43.664512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 73482d710c504010a4476f2f455b7b89. Sep 11 05:17:43.664512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:43.666683 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 462/947] 10.222.0.33 () {60 vars in 1140 bytes} [Sun Sep 11 05:17:43 2022] POST /identity/v3/projects => generated 262 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:43.673605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c2888-bb91-4de8-89a7-8d4f44e0dc44 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:43.684344 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:43.780337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c2888-bb91-4de8-89a7-8d4f44e0dc44 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:43.818586 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950bc68-5467-47e6-af83-6ad7b3f79c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:43.819484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950bc68-5467-47e6-af83-6ad7b3f79c8c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:43.819905 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950bc68-5467-47e6-af83-6ad7b3f79c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:43.820342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950bc68-5467-47e6-af83-6ad7b3f79c8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:43.827360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950bc68-5467-47e6-af83-6ad7b3f79c8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:43.829084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950bc68-5467-47e6-af83-6ad7b3f79c8c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-9950bc68-5467-47e6-af83-6ad7b3f79c8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:43.840534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:43.841583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:43.842679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:43.842679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:43.842679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:43.842679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:43.842679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:43.842679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:43.842679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:43.842679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:43.845193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 486/948] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:43.856126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714529c8-d5ba-4078-a821-b04b801be750 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:43.875520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:43.966989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714529c8-d5ba-4078-a821-b04b801be750 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:44.007127 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3af1d9-aac8-4e7b-8e44-478889321c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:44.007737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3af1d9-aac8-4e7b-8e44-478889321c2c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:44.008027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3af1d9-aac8-4e7b-8e44-478889321c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:44.008306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3af1d9-aac8-4e7b-8e44-478889321c2c None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:44.008971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3af1d9-aac8-4e7b-8e44-478889321c2c None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:44.009269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3af1d9-aac8-4e7b-8e44-478889321c2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:44.010912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3af1d9-aac8-4e7b-8e44-478889321c2c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:44.032452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 463/949] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 05:17:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:44.094162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74f3e4-d7a7-4bf4-b33b-1f8dc89e5114 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:44.094928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74f3e4-d7a7-4bf4-b33b-1f8dc89e5114 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:44.094928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74f3e4-d7a7-4bf4-b33b-1f8dc89e5114 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:44.096607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 487/950] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:44.102594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeb101-3b9a-4760-a46c-4ef4269f882c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:44.103184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeb101-3b9a-4760-a46c-4ef4269f882c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:44.103333 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeb101-3b9a-4760-a46c-4ef4269f882c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:44.124061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-50eeb101-3b9a-4760-a46c-4ef4269f882c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:44.125820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-50eeb101-3b9a-4760-a46c-4ef4269f882c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:44.203056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 464/951] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:44.477452 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74f3e4-d7a7-4bf4-b33b-1f8dc89e5114 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:44.543952 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:44.587129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74f3e4-d7a7-4bf4-b33b-1f8dc89e5114 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:44.623209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058cfb76-86ec-44f9-bf80-9d659e640eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:44.624049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cfb76-86ec-44f9-bf80-9d659e640eef None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:44.624430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cfb76-86ec-44f9-bf80-9d659e640eef None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:44.624805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cfb76-86ec-44f9-bf80-9d659e640eef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:44.625487 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058cfb76-86ec-44f9-bf80-9d659e640eef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:44.627398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058cfb76-86ec-44f9-bf80-9d659e640eef None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-058cfb76-86ec-44f9-bf80-9d659e640eef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:44.636970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:44.638166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:44.639427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:44.642576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 488/952] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 05:17:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:44.647792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eeb101-3b9a-4760-a46c-4ef4269f882c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:44.660934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:44.759212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eeb101-3b9a-4760-a46c-4ef4269f882c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:44.799491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a3629-edfc-4027-89fc-fddb9addfba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:44.800336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a3629-edfc-4027-89fc-fddb9addfba0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:44.800713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a3629-edfc-4027-89fc-fddb9addfba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:44.801084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a3629-edfc-4027-89fc-fddb9addfba0 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:44.801850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a3629-edfc-4027-89fc-fddb9addfba0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:44.802278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a3629-edfc-4027-89fc-fddb9addfba0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:44.803880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a3629-edfc-4027-89fc-fddb9addfba0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:44.810548 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 465/953] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 05:17:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:44.817526 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058cfb76-86ec-44f9-bf80-9d659e640eef None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:44.927278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058cfb76-86ec-44f9-bf80-9d659e640eef None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:44.928982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c15e03-f21e-4e4f-a879-77206f8c5ea1 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:44.929263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c15e03-f21e-4e4f-a879-77206f8c5ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:44.929717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c15e03-f21e-4e4f-a879-77206f8c5ea1 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:44.932126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 466/954] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:44.941560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a58c40-db1d-42a1-ba08-b544e3d4f7d4 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:44.942373 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a58c40-db1d-42a1-ba08-b544e3d4f7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:44.942748 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a58c40-db1d-42a1-ba08-b544e3d4f7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:44.961943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:44.964146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d6a58c40-db1d-42a1-ba08-b544e3d4f7d4 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:44.965379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d6a58c40-db1d-42a1-ba08-b544e3d4f7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:44.973376 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e46a0d-a014-4a4a-9dec-6c3207f6f6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:44.974737 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e46a0d-a014-4a4a-9dec-6c3207f6f6ed None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:44.975026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e46a0d-a014-4a4a-9dec-6c3207f6f6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:44.975293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e46a0d-a014-4a4a-9dec-6c3207f6f6ed None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:44.975943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e46a0d-a014-4a4a-9dec-6c3207f6f6ed None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:44.976401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e46a0d-a014-4a4a-9dec-6c3207f6f6ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:44.977848 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e46a0d-a014-4a4a-9dec-6c3207f6f6ed None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:44.984845 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 489/955] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 05:17:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:44.992896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e46a0d-a014-4a4a-9dec-6c3207f6f6ed None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:45.054875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:45.068094 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 467/956] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:45.099579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d8459-9873-40c5-a439-28b5d56d95d9 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:45.100220 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d8459-9873-40c5-a439-28b5d56d95d9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:45.100717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d8459-9873-40c5-a439-28b5d56d95d9 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:45.102690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 468/957] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:45.109132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e47cbd-508d-4409-86e8-26a3bd045a42 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:45.109651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e47cbd-508d-4409-86e8-26a3bd045a42 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:45.110158 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e47cbd-508d-4409-86e8-26a3bd045a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:45.126970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e46a0d-a014-4a4a-9dec-6c3207f6f6ed None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:45.131346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b3e47cbd-508d-4409-86e8-26a3bd045a42 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:45.132753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b3e47cbd-508d-4409-86e8-26a3bd045a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:45.166048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7efcf-91f2-4f05-bf4d-76fad4fa8216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:45.166663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7efcf-91f2-4f05-bf4d-76fad4fa8216 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:45.166955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7efcf-91f2-4f05-bf4d-76fad4fa8216 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:45.167189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7efcf-91f2-4f05-bf4d-76fad4fa8216 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:45.167845 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7efcf-91f2-4f05-bf4d-76fad4fa8216 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:45.169246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7efcf-91f2-4f05-bf4d-76fad4fa8216 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:45.200053 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 490/958] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:17:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:17:45.221784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 469/959] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:45.347678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7efcf-91f2-4f05-bf4d-76fad4fa8216 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:45.401502 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:45.461083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7efcf-91f2-4f05-bf4d-76fad4fa8216 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:45.495919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8db409-a213-43d0-a8ad-4bd5ebc6c02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:45.496985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8db409-a213-43d0-a8ad-4bd5ebc6c02b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:45.497470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8db409-a213-43d0-a8ad-4bd5ebc6c02b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:45.497943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8db409-a213-43d0-a8ad-4bd5ebc6c02b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:45.498745 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8db409-a213-43d0-a8ad-4bd5ebc6c02b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:45.500339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8db409-a213-43d0-a8ad-4bd5ebc6c02b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-3b8db409-a213-43d0-a8ad-4bd5ebc6c02b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:45.510641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:45.511917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 05:17:45.512923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:45.513847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e47cbd-508d-4409-86e8-26a3bd045a42 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:45.515407 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 491/960] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:45.522299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8db409-a213-43d0-a8ad-4bd5ebc6c02b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:45.622591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e47cbd-508d-4409-86e8-26a3bd045a42 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:45.628447 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8db409-a213-43d0-a8ad-4bd5ebc6c02b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:45.639690 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:45.653833 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:45.664609 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87a220-d755-4f20-99b6-395ed840f534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:45.665313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87a220-d755-4f20-99b6-395ed840f534 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:45.665596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87a220-d755-4f20-99b6-395ed840f534 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:45.665864 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87a220-d755-4f20-99b6-395ed840f534 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:45.668632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55400dd-b548-471c-8122-10c4ae02444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:45.669402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55400dd-b548-471c-8122-10c4ae02444f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:45.669830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55400dd-b548-471c-8122-10c4ae02444f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:45.670247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55400dd-b548-471c-8122-10c4ae02444f None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:45.671035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55400dd-b548-471c-8122-10c4ae02444f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:45.671482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55400dd-b548-471c-8122-10c4ae02444f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:45.673066 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55400dd-b548-471c-8122-10c4ae02444f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:45.677104 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87a220-d755-4f20-99b6-395ed840f534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-8a87a220-d755-4f20-99b6-395ed840f534 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:45.683003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:45.684165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:45.685132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:45.685132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:45.685132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:45.685132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:45.685132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:45.685132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:45.685132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:45.685132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:45.685132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:45.685132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:45.685132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 05:17:45.685132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:45.685832 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 492/962] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 05:17:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:45.686812 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55400dd-b548-471c-8122-10c4ae02444f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:45.687916 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 470/962] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 11 05:17:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:45.689322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87a220-d755-4f20-99b6-395ed840f534 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:45.796434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55400dd-b548-471c-8122-10c4ae02444f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:45.805777 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87a220-d755-4f20-99b6-395ed840f534 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:45.834882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab1563-8804-44a0-ba2c-fb890c4074cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:45.835738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab1563-8804-44a0-ba2c-fb890c4074cc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:45.836201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab1563-8804-44a0-ba2c-fb890c4074cc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:45.836644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab1563-8804-44a0-ba2c-fb890c4074cc None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:45.837521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab1563-8804-44a0-ba2c-fb890c4074cc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:45.838071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab1563-8804-44a0-ba2c-fb890c4074cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:45.839595 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab1563-8804-44a0-ba2c-fb890c4074cc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:45.842986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc2e0b-0155-43a1-8ca5-072a4d8e493f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:45.843694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc2e0b-0155-43a1-8ca5-072a4d8e493f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:45.844057 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc2e0b-0155-43a1-8ca5-072a4d8e493f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:45.844448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc2e0b-0155-43a1-8ca5-072a4d8e493f None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:45.845817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 493/963] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 05:17:45 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:45.848180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc2e0b-0155-43a1-8ca5-072a4d8e493f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:45.848180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc2e0b-0155-43a1-8ca5-072a4d8e493f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:45.848180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc2e0b-0155-43a1-8ca5-072a4d8e493f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:45.857701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 471/964] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 05:17:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:45.865246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab1563-8804-44a0-ba2c-fb890c4074cc None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:45.912920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:45.978026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab1563-8804-44a0-ba2c-fb890c4074cc None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:46.016467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfc855-cacf-49f2-a938-ffa45bdc6d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:46.017484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfc855-cacf-49f2-a938-ffa45bdc6d47 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:46.017771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfc855-cacf-49f2-a938-ffa45bdc6d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:46.018039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfc855-cacf-49f2-a938-ffa45bdc6d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:46.030055 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfc855-cacf-49f2-a938-ffa45bdc6d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:46.031522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfc855-cacf-49f2-a938-ffa45bdc6d47 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-4cdfc855-cacf-49f2-a938-ffa45bdc6d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:46.050803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:46.051819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:46.053357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:46.053357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:46.053357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:46.053357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 494/965] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:46.058017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc2e0b-0155-43a1-8ca5-072a4d8e493f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:46.116383 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:46.167923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc2e0b-0155-43a1-8ca5-072a4d8e493f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:46.206921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5cd17-0a93-462c-875d-d70a34f399f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:46.207535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5cd17-0a93-462c-875d-d70a34f399f9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:46.207812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5cd17-0a93-462c-875d-d70a34f399f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:46.208118 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5cd17-0a93-462c-875d-d70a34f399f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:46.208763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5cd17-0a93-462c-875d-d70a34f399f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:46.209061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5cd17-0a93-462c-875d-d70a34f399f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:46.210421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5cd17-0a93-462c-875d-d70a34f399f9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:46.223309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 472/966] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:46.228869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfc855-cacf-49f2-a938-ffa45bdc6d47 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:46.327566 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:46.338013 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfc855-cacf-49f2-a938-ffa45bdc6d47 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:46.373843 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c47177-b86b-43fd-a625-1e41812530bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:46.374477 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c47177-b86b-43fd-a625-1e41812530bc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:46.374753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c47177-b86b-43fd-a625-1e41812530bc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:46.375024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c47177-b86b-43fd-a625-1e41812530bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:46.383441 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c47177-b86b-43fd-a625-1e41812530bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:46.384914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c47177-b86b-43fd-a625-1e41812530bc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:46.394235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5909b12-1b8b-4f59-a0fc-d5b99792e7a8 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:46.394526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5909b12-1b8b-4f59-a0fc-d5b99792e7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:46.394804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5909b12-1b8b-4f59-a0fc-d5b99792e7a8 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:46.396559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 473/967] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-59c47177-b86b-43fd-a625-1e41812530bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:46.398575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:46.399524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:46.401253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:46.401253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:46.401253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:46.401253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:46.401253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:46.401253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:46.401253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:46.401253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:46.401253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 05:17:46.401253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:46.401253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 495/968] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 05:17:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:46.401876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c376439-7cc9-4acf-837f-4f53843d6981 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:46.401876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c376439-7cc9-4acf-837f-4f53843d6981 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:46.402196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c376439-7cc9-4acf-837f-4f53843d6981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:46.405439 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c47177-b86b-43fd-a625-1e41812530bc None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:46.421341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3c376439-7cc9-4acf-837f-4f53843d6981 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:46.422570 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3c376439-7cc9-4acf-837f-4f53843d6981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:46.507289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 474/969] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:46.520343 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c47177-b86b-43fd-a625-1e41812530bc None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:46.555237 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011eccd-e23f-4f89-827d-d0f20157b5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:46.555842 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011eccd-e23f-4f89-827d-d0f20157b5ef None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:46.556147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011eccd-e23f-4f89-827d-d0f20157b5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:46.556416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011eccd-e23f-4f89-827d-d0f20157b5ef None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:46.557143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011eccd-e23f-4f89-827d-d0f20157b5ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:46.557442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011eccd-e23f-4f89-827d-d0f20157b5ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:46.558865 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011eccd-e23f-4f89-827d-d0f20157b5ef None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:46.577186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 496/970] 10.222.0.33 () {56 vars in 1149 bytes} [Sun Sep 11 05:17:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:46.583413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c376439-7cc9-4acf-837f-4f53843d6981 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:46.626630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:46.691256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c376439-7cc9-4acf-837f-4f53843d6981 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:46.727745 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1552d3b-5d62-4c76-8000-d1ee1957b56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:46.728385 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1552d3b-5d62-4c76-8000-d1ee1957b56f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:46.728667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1552d3b-5d62-4c76-8000-d1ee1957b56f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:46.728941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1552d3b-5d62-4c76-8000-d1ee1957b56f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:46.785922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011eccd-e23f-4f89-827d-d0f20157b5ef None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:46.801263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1552d3b-5d62-4c76-8000-d1ee1957b56f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:46.802739 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1552d3b-5d62-4c76-8000-d1ee1957b56f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:46.808250 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:46.849038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:46.915839 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011eccd-e23f-4f89-827d-d0f20157b5ef None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:46.930045 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 475/971] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:46 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:17:46.935268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1552d3b-5d62-4c76-8000-d1ee1957b56f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:46.955798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecbafd8-d51b-48ad-84a5-bbe9d38e3c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:46.956726 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbafd8-d51b-48ad-84a5-bbe9d38e3c6b None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:46.957238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbafd8-d51b-48ad-84a5-bbe9d38e3c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:46.957731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbafd8-d51b-48ad-84a5-bbe9d38e3c6b None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:46.958574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecbafd8-d51b-48ad-84a5-bbe9d38e3c6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:46.960292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecbafd8-d51b-48ad-84a5-bbe9d38e3c6b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:46.963030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: WARNING keystone.server.flask.common [None req-0ecbafd8-d51b-48ad-84a5-bbe9d38e3c6b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 11 05:17:46.977436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.sql.core [None req-0ecbafd8-d51b-48ad-84a5-bbe9d38e3c6b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 05:17:46.977436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 11 05:17:46.977436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [parameters: {'user_id': '6eba9b53cd434d51bb84554849a8f349', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 11 05:17:46.977436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81444) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-0ecbafd8-d51b-48ad-84a5-bbe9d38e3c6b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application conn.query(q) Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application result.read() Sep 11 05:17:46.996665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 05:17:46.997619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application next(self.gen) Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application yield resource Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application next(self.gen) Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise self.value Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application session.commit() Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 05:17:46.998629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.session.flush() Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application compat.raise_( Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 05:17:46.999537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 05:17:47.000642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application conn.query(q) Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application result.read() Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 05:17:47.001970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [parameters: {'user_id': '6eba9b53cd434d51bb84554849a8f349', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:47.005474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 11 05:17:47.007484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 11 05:17:47.009942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 11 05:17:47.009942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 11 05:17:47.009942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 05:17:47.009942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 05:17:47.009942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 11 05:17:47.009942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:47.009942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 497/972] 10.222.0.33 () {60 vars in 1131 bytes} [Sun Sep 11 05:17:46 2022] POST /identity/v3/users => generated 160 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:47.009942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecbafd8-d51b-48ad-84a5-bbe9d38e3c6b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:47.066601 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1552d3b-5d62-4c76-8000-d1ee1957b56f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:47.084959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:47.107948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6b71b-3401-4a97-b540-f8fe305add80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:47.109021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6b71b-3401-4a97-b540-f8fe305add80 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:47.109502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6b71b-3401-4a97-b540-f8fe305add80 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:47.110493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6b71b-3401-4a97-b540-f8fe305add80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:47.118502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:47.122669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6b71b-3401-4a97-b540-f8fe305add80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:47.123626 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecbafd8-d51b-48ad-84a5-bbe9d38e3c6b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:47.124535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6b71b-3401-4a97-b540-f8fe305add80 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-46c6b71b-3401-4a97-b540-f8fe305add80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:47.143029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:47.144151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:47.145085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:47.145085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:47.145085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:47.149225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 476/973] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:47.150145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6b71b-3401-4a97-b540-f8fe305add80 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:47.166570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a66b9-8640-4832-aa34-26f4609fe149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:47.167491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a66b9-8640-4832-aa34-26f4609fe149 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:47.168103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a66b9-8640-4832-aa34-26f4609fe149 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:47.168632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a66b9-8640-4832-aa34-26f4609fe149 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:47.177196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a66b9-8640-4832-aa34-26f4609fe149 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:47.179279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a66b9-8640-4832-aa34-26f4609fe149 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-0c9a66b9-8640-4832-aa34-26f4609fe149 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:47.191239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:47.192268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:47.193227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:47.193227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:47.193227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:47.193227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:47.193227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:47.193227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:47.193227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:47.193227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:47.193227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 05:17:47.193227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:47.195956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 498/974] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:17:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:47.209038 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a66b9-8640-4832-aa34-26f4609fe149 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:47.264295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6b71b-3401-4a97-b540-f8fe305add80 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:47.300902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bae79-f146-42fe-8547-bf8d16fcec34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:47.302136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bae79-f146-42fe-8547-bf8d16fcec34 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:47.302652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bae79-f146-42fe-8547-bf8d16fcec34 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:47.303093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bae79-f146-42fe-8547-bf8d16fcec34 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:47.303842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bae79-f146-42fe-8547-bf8d16fcec34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:47.304389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bae79-f146-42fe-8547-bf8d16fcec34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:47.306655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bae79-f146-42fe-8547-bf8d16fcec34 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:47.327736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 477/975] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:47.328783 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bae79-f146-42fe-8547-bf8d16fcec34 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:47.344828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a66b9-8640-4832-aa34-26f4609fe149 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:47.386326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:47.389062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:47.397994 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3078654-6ff4-49be-b8c7-2c57e458508d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:47.398622 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3078654-6ff4-49be-b8c7-2c57e458508d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:47.398897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3078654-6ff4-49be-b8c7-2c57e458508d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:47.399160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3078654-6ff4-49be-b8c7-2c57e458508d None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:47.400048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3078654-6ff4-49be-b8c7-2c57e458508d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:47.400354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3078654-6ff4-49be-b8c7-2c57e458508d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:47.401841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3078654-6ff4-49be-b8c7-2c57e458508d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:47.444043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 499/976] 10.222.0.33 () {56 vars in 1101 bytes} [Sun Sep 11 05:17:47 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:47.472756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bae79-f146-42fe-8547-bf8d16fcec34 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:47.511898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdf99f-0310-400d-9d0b-e799d73e6acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:47.512877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdf99f-0310-400d-9d0b-e799d73e6acc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:47.513307 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdf99f-0310-400d-9d0b-e799d73e6acc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:47.513725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdf99f-0310-400d-9d0b-e799d73e6acc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:47.522475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bdf99f-0310-400d-9d0b-e799d73e6acc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:47.523375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732d6ec-1485-49fe-904e-aeb8645dc13e None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:47.523662 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732d6ec-1485-49fe-904e-aeb8645dc13e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:47.523937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732d6ec-1485-49fe-904e-aeb8645dc13e None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:47.524903 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bdf99f-0310-400d-9d0b-e799d73e6acc None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:47.529733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 500/977] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:47.531430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eb213-8816-4a17-9568-a98a7d6a3c2a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:47.531717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eb213-8816-4a17-9568-a98a7d6a3c2a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:47.532568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eb213-8816-4a17-9568-a98a7d6a3c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-00bdf99f-0310-400d-9d0b-e799d73e6acc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:47.542623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:47.543598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:47.549189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:47.549189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:47.549189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:47.549189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:47.549189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:47.549189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:47.549189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:47.549189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:47.549189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 478/978] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:47.552315 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdf99f-0310-400d-9d0b-e799d73e6acc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:47.570051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-853eb213-8816-4a17-9568-a98a7d6a3c2a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:47.571753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-853eb213-8816-4a17-9568-a98a7d6a3c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:47.655540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 501/979] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:47.685394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdf99f-0310-400d-9d0b-e799d73e6acc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:47.700126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:47.724688 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d4459-e46b-46e0-8bfb-a7c3134fbcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:47.725278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d4459-e46b-46e0-8bfb-a7c3134fbcac None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:47.725561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d4459-e46b-46e0-8bfb-a7c3134fbcac None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:47.725870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d4459-e46b-46e0-8bfb-a7c3134fbcac None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:47.726566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d4459-e46b-46e0-8bfb-a7c3134fbcac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:47.726869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d4459-e46b-46e0-8bfb-a7c3134fbcac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:47.728355 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d4459-e46b-46e0-8bfb-a7c3134fbcac None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:47.742360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 479/980] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:47.748462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853eb213-8816-4a17-9568-a98a7d6a3c2a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:47.855188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853eb213-8816-4a17-9568-a98a7d6a3c2a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:47.872793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:47.890269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a641f3-55c2-4496-b735-8c6d48a5f13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:47.890869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a641f3-55c2-4496-b735-8c6d48a5f13b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:47.891143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a641f3-55c2-4496-b735-8c6d48a5f13b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:47.892195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a641f3-55c2-4496-b735-8c6d48a5f13b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:47.893051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a641f3-55c2-4496-b735-8c6d48a5f13b None admin] RBAC: Adding query filter params (role.id=a1192a1e2adb45d490e8d553aa87c6a6, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=afd0a6d3b2194604a46767cfde12097b) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:47.893359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a641f3-55c2-4496-b735-8c6d48a5f13b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:47.894779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a641f3-55c2-4496-b735-8c6d48a5f13b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:47.908078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 502/981] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:17:47 2022] GET /identity/v3/role_assignments?user.id=afd0a6d3b2194604a46767cfde12097b&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=a1192a1e2adb45d490e8d553aa87c6a6 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:47.933994 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d4459-e46b-46e0-8bfb-a7c3134fbcac None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:48.046794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d4459-e46b-46e0-8bfb-a7c3134fbcac None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:48.090083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8fc0f-2de3-4205-97f8-1af26e3619ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:48.090734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8fc0f-2de3-4205-97f8-1af26e3619ac None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:48.091010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8fc0f-2de3-4205-97f8-1af26e3619ac None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:48.091276 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8fc0f-2de3-4205-97f8-1af26e3619ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:48.091866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8fc0f-2de3-4205-97f8-1af26e3619ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:48.093282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8fc0f-2de3-4205-97f8-1af26e3619ac None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-11c8fc0f-2de3-4205-97f8-1af26e3619ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:48.103630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:48.104713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:48.105572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 480/982] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 05:17:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:48.110965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a641f3-55c2-4496-b735-8c6d48a5f13b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:48.183932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffb2dd-7f1f-4d1e-979a-592f62b51d0f None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:48.184482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffb2dd-7f1f-4d1e-979a-592f62b51d0f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:48.184572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffb2dd-7f1f-4d1e-979a-592f62b51d0f None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:48.185850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 481/983] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:48.191346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fe0c8-6902-411b-8df1-71c1fd9107a0 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:48.191635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fe0c8-6902-411b-8df1-71c1fd9107a0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:48.191913 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fe0c8-6902-411b-8df1-71c1fd9107a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:48.209899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-632fe0c8-6902-411b-8df1-71c1fd9107a0 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:48.211101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-632fe0c8-6902-411b-8df1-71c1fd9107a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:48.221608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a641f3-55c2-4496-b735-8c6d48a5f13b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:48.239297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:48.260156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f45e8-ceda-462e-b034-01650a29dbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:48.260766 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f45e8-ceda-462e-b034-01650a29dbfd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:48.261043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f45e8-ceda-462e-b034-01650a29dbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:48.261306 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f45e8-ceda-462e-b034-01650a29dbfd None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:48.261929 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f45e8-ceda-462e-b034-01650a29dbfd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:48.262229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f45e8-ceda-462e-b034-01650a29dbfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:48.263603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f45e8-ceda-462e-b034-01650a29dbfd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:48.268981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 503/984] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 05:17:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:48.274395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f45e8-ceda-462e-b034-01650a29dbfd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:48.283433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 482/985] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:48.322901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:48.379029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f45e8-ceda-462e-b034-01650a29dbfd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:48.412848 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c784e7-8070-4cf7-aa53-418978fc4536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:48.413453 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c784e7-8070-4cf7-aa53-418978fc4536 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:48.413730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c784e7-8070-4cf7-aa53-418978fc4536 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:48.413996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c784e7-8070-4cf7-aa53-418978fc4536 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:48.414620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c784e7-8070-4cf7-aa53-418978fc4536 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:48.414919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c784e7-8070-4cf7-aa53-418978fc4536 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:48.417392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c784e7-8070-4cf7-aa53-418978fc4536 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:48.423217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 504/986] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 05:17:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:48.429019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fe0c8-6902-411b-8df1-71c1fd9107a0 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:48.534084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fe0c8-6902-411b-8df1-71c1fd9107a0 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:48.554149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c784e7-8070-4cf7-aa53-418978fc4536 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:48.571019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7ddb4-aa8e-4014-b728-fb8c0f1ce98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:48.571634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7ddb4-aa8e-4014-b728-fb8c0f1ce98d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:48.571899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7ddb4-aa8e-4014-b728-fb8c0f1ce98d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:48.572199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7ddb4-aa8e-4014-b728-fb8c0f1ce98d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:48.572970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7ddb4-aa8e-4014-b728-fb8c0f1ce98d None admin] RBAC: Adding query filter params (interface=admin, service_id=4996407a6ae94e0791fc4e0633a3cd40, region_id=RegionOne) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:48.573212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7ddb4-aa8e-4014-b728-fb8c0f1ce98d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:48.574583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7ddb4-aa8e-4014-b728-fb8c0f1ce98d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:48.581078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 483/987] 10.222.0.33 () {56 vars in 1249 bytes} [Sun Sep 11 05:17:48 2022] GET /identity/v3/endpoints?service_id=4996407a6ae94e0791fc4e0633a3cd40&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:48.586427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7ddb4-aa8e-4014-b728-fb8c0f1ce98d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:48.615522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:48.662857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c784e7-8070-4cf7-aa53-418978fc4536 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:48.673985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:48.693715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7ddb4-aa8e-4014-b728-fb8c0f1ce98d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:48.699500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b192fe-0ae1-47f5-bd2a-660f23c88a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:48.700158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b192fe-0ae1-47f5-bd2a-660f23c88a8f None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:48.700436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b192fe-0ae1-47f5-bd2a-660f23c88a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:48.700704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b192fe-0ae1-47f5-bd2a-660f23c88a8f None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:48.701463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b192fe-0ae1-47f5-bd2a-660f23c88a8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:48.702809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b192fe-0ae1-47f5-bd2a-660f23c88a8f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:48.720177 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 505/988] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:17:48 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:17:48.738336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb1a5b-0b8d-4509-bb5c-ea68b4c24a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:48.738974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb1a5b-0b8d-4509-bb5c-ea68b4c24a92 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:48.739256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb1a5b-0b8d-4509-bb5c-ea68b4c24a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:48.739530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb1a5b-0b8d-4509-bb5c-ea68b4c24a92 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:48.740243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb1a5b-0b8d-4509-bb5c-ea68b4c24a92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:48.741802 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb1a5b-0b8d-4509-bb5c-ea68b4c24a92 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:48.760034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 484/989] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 05:17:48 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:17:50.112168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076c437-e688-4e4c-86f5-b4ff9f6a632f None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:50.112686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076c437-e688-4e4c-86f5-b4ff9f6a632f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:50.112686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076c437-e688-4e4c-86f5-b4ff9f6a632f None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:50.114208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 506/990] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:50.119321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59327bf8-d926-4abd-9861-541ec3b15c74 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:50.119715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59327bf8-d926-4abd-9861-541ec3b15c74 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:50.119878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59327bf8-d926-4abd-9861-541ec3b15c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:50.138714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-59327bf8-d926-4abd-9861-541ec3b15c74 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:50.139895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-59327bf8-d926-4abd-9861-541ec3b15c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:50.208917 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 485/991] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:50.474142 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076c437-e688-4e4c-86f5-b4ff9f6a632f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:50.578154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076c437-e688-4e4c-86f5-b4ff9f6a632f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:50.612960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a29116-63b1-461f-9226-9d857418f9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:50.613592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a29116-63b1-461f-9226-9d857418f9cb None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:50.613871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a29116-63b1-461f-9226-9d857418f9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:50.614148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a29116-63b1-461f-9226-9d857418f9cb None admin] PATH_INFO: `/v3/services/image` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:50.614724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a29116-63b1-461f-9226-9d857418f9cb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:50.616119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a29116-63b1-461f-9226-9d857418f9cb None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-94a29116-63b1-461f-9226-9d857418f9cb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:50.624814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:50.626018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 05:17:50.627104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:50.631011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 507/992] 10.222.0.33 () {56 vars in 1105 bytes} [Sun Sep 11 05:17:50 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:50.632427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59327bf8-d926-4abd-9861-541ec3b15c74 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:50.650888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:50.734710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59327bf8-d926-4abd-9861-541ec3b15c74 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:50.772564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38062a0-dc6e-44e0-932e-fde6564605bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:50.773173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38062a0-dc6e-44e0-932e-fde6564605bb None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:50.773450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38062a0-dc6e-44e0-932e-fde6564605bb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:50.773718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38062a0-dc6e-44e0-932e-fde6564605bb None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:50.774348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38062a0-dc6e-44e0-932e-fde6564605bb None admin] RBAC: Adding query filter params (name=image) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:50.774649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38062a0-dc6e-44e0-932e-fde6564605bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:50.776032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38062a0-dc6e-44e0-932e-fde6564605bb None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:50.781360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 486/993] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:50.787543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a29116-63b1-461f-9226-9d857418f9cb None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:50.828585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:50.902821 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a29116-63b1-461f-9226-9d857418f9cb None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:50.924510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc6215-d3b9-4077-b39f-bf22280bee29 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:50.924835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc6215-d3b9-4077-b39f-bf22280bee29 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:50.925064 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc6215-d3b9-4077-b39f-bf22280bee29 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:50.926575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 487/994] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:50.931345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdff6f-50ee-4856-8463-0d979e57e6de None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:50.931626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdff6f-50ee-4856-8463-0d979e57e6de None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:50.931898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdff6f-50ee-4856-8463-0d979e57e6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:50.938573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf4d2a-80d7-40b0-aae0-097c71d0f626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:50.940072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf4d2a-80d7-40b0-aae0-097c71d0f626 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:50.940373 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf4d2a-80d7-40b0-aae0-097c71d0f626 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:50.940646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf4d2a-80d7-40b0-aae0-097c71d0f626 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:50.941283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf4d2a-80d7-40b0-aae0-097c71d0f626 None admin] RBAC: Adding query filter params (type=image) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:50.941586 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf4d2a-80d7-40b0-aae0-097c71d0f626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:50.942954 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf4d2a-80d7-40b0-aae0-097c71d0f626 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:50.947966 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 508/995] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:50.950312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f4bdff6f-50ee-4856-8463-0d979e57e6de None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:50.951530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f4bdff6f-50ee-4856-8463-0d979e57e6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:50.962042 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c4e34-beb2-450e-a1a3-e8c30cba02aa None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:50.962408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c4e34-beb2-450e-a1a3-e8c30cba02aa None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:50.962653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c4e34-beb2-450e-a1a3-e8c30cba02aa None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:50.964263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 509/996] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:50.971179 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a1fa7-4d59-4884-868a-1c79e77ea3f9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:50.971527 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a1fa7-4d59-4884-868a-1c79e77ea3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:50.971729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a1fa7-4d59-4884-868a-1c79e77ea3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:50.992569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1a8a1fa7-4d59-4884-868a-1c79e77ea3f9 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:50.993727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1a8a1fa7-4d59-4884-868a-1c79e77ea3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:51.032644 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 488/997] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:51.080178 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 510/998] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:51.311888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdff6f-50ee-4856-8463-0d979e57e6de None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:51.326089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:51.356266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a1fa7-4d59-4884-868a-1c79e77ea3f9 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:51.386749 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:51.418453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdff6f-50ee-4856-8463-0d979e57e6de None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:51.457963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec65c8e-b374-432e-96c8-35cfca4c343f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:51.458598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec65c8e-b374-432e-96c8-35cfca4c343f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:51.458880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec65c8e-b374-432e-96c8-35cfca4c343f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:51.459212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec65c8e-b374-432e-96c8-35cfca4c343f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:51.459729 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec65c8e-b374-432e-96c8-35cfca4c343f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:51.461146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec65c8e-b374-432e-96c8-35cfca4c343f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:51.464372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a1fa7-4d59-4884-868a-1c79e77ea3f9 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-7ec65c8e-b374-432e-96c8-35cfca4c343f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:51.470941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:51.471951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:51.472834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 489/999] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:51.478173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec65c8e-b374-432e-96c8-35cfca4c343f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:51.503727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b358882-94fb-47bf-b8bf-8357ce5ad247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:51.504937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b358882-94fb-47bf-b8bf-8357ce5ad247 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:51.505223 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b358882-94fb-47bf-b8bf-8357ce5ad247 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:51.505497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b358882-94fb-47bf-b8bf-8357ce5ad247 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:51.506081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b358882-94fb-47bf-b8bf-8357ce5ad247 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:51.507442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b358882-94fb-47bf-b8bf-8357ce5ad247 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-2b358882-94fb-47bf-b8bf-8357ce5ad247 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:51.516836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:51.518158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:51.519014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 511/1000] 10.222.0.33 () {56 vars in 1126 bytes} [Sun Sep 11 05:17:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:51.523755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b358882-94fb-47bf-b8bf-8357ce5ad247 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:51.566491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:51.591610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec65c8e-b374-432e-96c8-35cfca4c343f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:51.624226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:51.629537 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08c6d1-86ea-43da-ba86-ec3f9cf54b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:51.630146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08c6d1-86ea-43da-ba86-ec3f9cf54b10 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:51.630421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08c6d1-86ea-43da-ba86-ec3f9cf54b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:51.630687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08c6d1-86ea-43da-ba86-ec3f9cf54b10 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:51.631315 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08c6d1-86ea-43da-ba86-ec3f9cf54b10 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:51.631609 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08c6d1-86ea-43da-ba86-ec3f9cf54b10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:51.633011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08c6d1-86ea-43da-ba86-ec3f9cf54b10 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:51.635495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b358882-94fb-47bf-b8bf-8357ce5ad247 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:51.642829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 490/1001] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 05:17:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:51.644185 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08c6d1-86ea-43da-ba86-ec3f9cf54b10 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:51.671775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c289689-9c07-40d1-b5e3-ee8b89e942a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:51.673006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c289689-9c07-40d1-b5e3-ee8b89e942a7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:51.673290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c289689-9c07-40d1-b5e3-ee8b89e942a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:51.673559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c289689-9c07-40d1-b5e3-ee8b89e942a7 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:51.674190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c289689-9c07-40d1-b5e3-ee8b89e942a7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:51.674495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c289689-9c07-40d1-b5e3-ee8b89e942a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:51.675918 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c289689-9c07-40d1-b5e3-ee8b89e942a7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:51.681486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 512/1002] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 05:17:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:51.686935 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c289689-9c07-40d1-b5e3-ee8b89e942a7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:51.749674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08c6d1-86ea-43da-ba86-ec3f9cf54b10 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:51.783508 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22b3cc-b50e-4d2e-9e75-2552e457017a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:51.784138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22b3cc-b50e-4d2e-9e75-2552e457017a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:51.784421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22b3cc-b50e-4d2e-9e75-2552e457017a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:51.784688 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22b3cc-b50e-4d2e-9e75-2552e457017a None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:51.785303 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22b3cc-b50e-4d2e-9e75-2552e457017a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:51.785599 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22b3cc-b50e-4d2e-9e75-2552e457017a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:51.786945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22b3cc-b50e-4d2e-9e75-2552e457017a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:51.796455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 491/1003] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 05:17:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:51.797984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22b3cc-b50e-4d2e-9e75-2552e457017a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:51.801551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:51.806955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c289689-9c07-40d1-b5e3-ee8b89e942a7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:51.845862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44144fa3-6fba-4e65-af93-33f3968cad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:51.846395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44144fa3-6fba-4e65-af93-33f3968cad71 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:51.846668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44144fa3-6fba-4e65-af93-33f3968cad71 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:51.846936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44144fa3-6fba-4e65-af93-33f3968cad71 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:51.847705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44144fa3-6fba-4e65-af93-33f3968cad71 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:51.848031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44144fa3-6fba-4e65-af93-33f3968cad71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:51.849534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44144fa3-6fba-4e65-af93-33f3968cad71 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:51.859559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 513/1004] 10.222.0.33 () {56 vars in 1122 bytes} [Sun Sep 11 05:17:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:51.861320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44144fa3-6fba-4e65-af93-33f3968cad71 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:51.886435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:51.913673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22b3cc-b50e-4d2e-9e75-2552e457017a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:51.949509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5752293-f7b9-4ce6-89ae-5a171c774830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:51.950401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5752293-f7b9-4ce6-89ae-5a171c774830 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:51.950932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5752293-f7b9-4ce6-89ae-5a171c774830 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:51.950932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5752293-f7b9-4ce6-89ae-5a171c774830 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:51.951701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5752293-f7b9-4ce6-89ae-5a171c774830 None admin] RBAC: Adding query filter params (interface=public, service_id=b7c46ec511e247449a8ec6b4bd15ca59, region_id=RegionOne) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:51.951922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5752293-f7b9-4ce6-89ae-5a171c774830 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:51.953326 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5752293-f7b9-4ce6-89ae-5a171c774830 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:51.963505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 492/1005] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 05:17:51 2022] GET /identity/v3/endpoints?service_id=b7c46ec511e247449a8ec6b4bd15ca59&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:51.965090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5752293-f7b9-4ce6-89ae-5a171c774830 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:51.971753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44144fa3-6fba-4e65-af93-33f3968cad71 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:52.007692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9b683-4b77-4c37-b4ee-c8b49b330029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:52.008331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9b683-4b77-4c37-b4ee-c8b49b330029 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:52.008612 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9b683-4b77-4c37-b4ee-c8b49b330029 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:52.008885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9b683-4b77-4c37-b4ee-c8b49b330029 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:52.009542 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9b683-4b77-4c37-b4ee-c8b49b330029 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:52.010939 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9b683-4b77-4c37-b4ee-c8b49b330029 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:52.040025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 514/1006] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:17:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:17:52.046989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:52.087751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5752293-f7b9-4ce6-89ae-5a171c774830 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:52.133056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a235340-24a4-40c8-8e20-e30e08470d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:52.133595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a235340-24a4-40c8-8e20-e30e08470d0a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:52.133874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a235340-24a4-40c8-8e20-e30e08470d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:52.134144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a235340-24a4-40c8-8e20-e30e08470d0a None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:52.134751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a235340-24a4-40c8-8e20-e30e08470d0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:52.136167 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a235340-24a4-40c8-8e20-e30e08470d0a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:52.143608 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 493/1007] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 05:17:51 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:17:53.148731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b32687-76bb-4913-aed6-0d5ba3c1f105 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:53.149281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b32687-76bb-4913-aed6-0d5ba3c1f105 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:53.149281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b32687-76bb-4913-aed6-0d5ba3c1f105 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:53.150642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 515/1008] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:53.155707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82af646-86bb-40aa-8023-1ee6dfc4052a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:53.156145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82af646-86bb-40aa-8023-1ee6dfc4052a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:53.156308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82af646-86bb-40aa-8023-1ee6dfc4052a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:53.174785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a82af646-86bb-40aa-8023-1ee6dfc4052a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:53.175968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a82af646-86bb-40aa-8023-1ee6dfc4052a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:53.242722 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 494/1009] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:53 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:53.509726 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b32687-76bb-4913-aed6-0d5ba3c1f105 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:53.592089 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:53.615361 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b32687-76bb-4913-aed6-0d5ba3c1f105 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:53.648782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d993dd-a6bb-4fc1-b9dc-d8230ddfd3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:53.649414 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d993dd-a6bb-4fc1-b9dc-d8230ddfd3b5 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:53.649695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d993dd-a6bb-4fc1-b9dc-d8230ddfd3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:53.649961 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d993dd-a6bb-4fc1-b9dc-d8230ddfd3b5 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:53.650609 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d993dd-a6bb-4fc1-b9dc-d8230ddfd3b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:53.651966 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d993dd-a6bb-4fc1-b9dc-d8230ddfd3b5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:53.667602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 516/1010] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:17:53 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:17:54.229227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e986-24d4-458a-a85d-bcb113232047 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:54.229784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e986-24d4-458a-a85d-bcb113232047 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:54.229784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e986-24d4-458a-a85d-bcb113232047 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:54.231345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 495/1011] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:54.237569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5662a22-861a-46e9-96b3-e1e8e8b2029a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:54.238119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5662a22-861a-46e9-96b3-e1e8e8b2029a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:54.238119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5662a22-861a-46e9-96b3-e1e8e8b2029a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:54.257652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b5662a22-861a-46e9-96b3-e1e8e8b2029a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:54.258832 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b5662a22-861a-46e9-96b3-e1e8e8b2029a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:54.334762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 517/1012] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:54 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:54.340055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cd3da-e741-4f27-9d84-0f9dbee41bbe None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:54.340347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cd3da-e741-4f27-9d84-0f9dbee41bbe None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:54.340621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cd3da-e741-4f27-9d84-0f9dbee41bbe None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:54.342067 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 496/1013] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:54.347084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec12335-b632-4c79-bbbf-cb67385701f2 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:54.347374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec12335-b632-4c79-bbbf-cb67385701f2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:54.347643 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec12335-b632-4c79-bbbf-cb67385701f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:54.366377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4ec12335-b632-4c79-bbbf-cb67385701f2 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:54.367563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4ec12335-b632-4c79-bbbf-cb67385701f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:54.429687 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:54.439165 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 518/1014] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:54 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:54.602763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cd3da-e741-4f27-9d84-0f9dbee41bbe None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:54.664406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:54.708071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec12335-b632-4c79-bbbf-cb67385701f2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:54.709961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cd3da-e741-4f27-9d84-0f9dbee41bbe None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:54.753168 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da316d-a0f8-477b-8085-834e7c7f6fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:54.754599 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da316d-a0f8-477b-8085-834e7c7f6fda None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:54.755405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da316d-a0f8-477b-8085-834e7c7f6fda None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:54.756171 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da316d-a0f8-477b-8085-834e7c7f6fda None admin] PATH_INFO: `/v3/domains/default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:54.757641 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da316d-a0f8-477b-8085-834e7c7f6fda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:54.759561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da316d-a0f8-477b-8085-834e7c7f6fda None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:54.762308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 497/1015] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:54.768543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da316d-a0f8-477b-8085-834e7c7f6fda None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:54.810291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec12335-b632-4c79-bbbf-cb67385701f2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:54.847389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af04ce5-d96c-49e3-81c3-fe868f968da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:54.848048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af04ce5-d96c-49e3-81c3-fe868f968da0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:54.848380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af04ce5-d96c-49e3-81c3-fe868f968da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:54.848580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af04ce5-d96c-49e3-81c3-fe868f968da0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:54.849152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af04ce5-d96c-49e3-81c3-fe868f968da0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:54.850506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af04ce5-d96c-49e3-81c3-fe868f968da0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-9af04ce5-d96c-49e3-81c3-fe868f968da0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:54.859317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:54.860371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:54.865540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 519/1016] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:54.866353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af04ce5-d96c-49e3-81c3-fe868f968da0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:54.902046 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da316d-a0f8-477b-8085-834e7c7f6fda None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:54.929866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:54.932246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:54.942003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9cb7d-259a-42d1-9ed3-d0e5112f48be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:54.942806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9cb7d-259a-42d1-9ed3-d0e5112f48be None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:54.943080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9cb7d-259a-42d1-9ed3-d0e5112f48be None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:54.943345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9cb7d-259a-42d1-9ed3-d0e5112f48be None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:54.944016 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9cb7d-259a-42d1-9ed3-d0e5112f48be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:54.945440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9cb7d-259a-42d1-9ed3-d0e5112f48be None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:54.980517 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af04ce5-d96c-49e3-81c3-fe868f968da0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:54.988400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 498/1017] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:17:54 2022] POST /identity/v3/projects => generated 311 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:17:55.020070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7bb55-8bac-44b3-af1e-0ee6b8a39a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:55.020691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7bb55-8bac-44b3-af1e-0ee6b8a39a44 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:55.020968 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7bb55-8bac-44b3-af1e-0ee6b8a39a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:55.021236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7bb55-8bac-44b3-af1e-0ee6b8a39a44 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:55.021871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7bb55-8bac-44b3-af1e-0ee6b8a39a44 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:55.022174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7bb55-8bac-44b3-af1e-0ee6b8a39a44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:55.023588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7bb55-8bac-44b3-af1e-0ee6b8a39a44 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:55.029456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 520/1018] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 05:17:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:55.035638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9cb7d-259a-42d1-9ed3-d0e5112f48be None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:55.146459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9cb7d-259a-42d1-9ed3-d0e5112f48be None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:55.189900 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddad38-0c05-4744-bc35-b081656e1f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:55.190516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddad38-0c05-4744-bc35-b081656e1f0e None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:55.191573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddad38-0c05-4744-bc35-b081656e1f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:55.191573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddad38-0c05-4744-bc35-b081656e1f0e None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:55.191848 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddad38-0c05-4744-bc35-b081656e1f0e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:55.192213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddad38-0c05-4744-bc35-b081656e1f0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:55.193464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddad38-0c05-4744-bc35-b081656e1f0e None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:55.198754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 499/1019] 10.222.0.33 () {56 vars in 1112 bytes} [Sun Sep 11 05:17:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:55.204702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7bb55-8bac-44b3-af1e-0ee6b8a39a44 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:55.310131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7bb55-8bac-44b3-af1e-0ee6b8a39a44 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:55.335532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:55.347816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be903766-02da-4dd4-a565-4dd3f5760e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:55.348455 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be903766-02da-4dd4-a565-4dd3f5760e2f None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:55.348736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be903766-02da-4dd4-a565-4dd3f5760e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:55.349006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be903766-02da-4dd4-a565-4dd3f5760e2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:55.349665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be903766-02da-4dd4-a565-4dd3f5760e2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:55.351026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be903766-02da-4dd4-a565-4dd3f5760e2f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:55.371012 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 521/1020] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:17:55 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:17:55.876969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a975a7-9416-4cc9-989a-1eaa17b7b4ed None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:55.877513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a975a7-9416-4cc9-989a-1eaa17b7b4ed None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:55.877594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a975a7-9416-4cc9-989a-1eaa17b7b4ed None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:55.879284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 500/1021] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:55.884312 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22044181-4c45-4679-b9e6-f4381b024ab0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:55.884600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22044181-4c45-4679-b9e6-f4381b024ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:55.884877 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22044181-4c45-4679-b9e6-f4381b024ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:55.903044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-22044181-4c45-4679-b9e6-f4381b024ab0 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:55.904273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-22044181-4c45-4679-b9e6-f4381b024ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:55.981310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 522/1022] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:56.248651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a975a7-9416-4cc9-989a-1eaa17b7b4ed None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:56.294773 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:56.359132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a975a7-9416-4cc9-989a-1eaa17b7b4ed None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:56.399968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e67501-428b-47df-9413-976adb5f149d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:56.400633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e67501-428b-47df-9413-976adb5f149d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:56.400913 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e67501-428b-47df-9413-976adb5f149d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:56.401188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e67501-428b-47df-9413-976adb5f149d None admin] PATH_INFO: `/v3/services/image` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:56.401765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e67501-428b-47df-9413-976adb5f149d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:56.403158 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e67501-428b-47df-9413-976adb5f149d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-45e67501-428b-47df-9413-976adb5f149d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:56.412135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:56.413297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 05:17:56.414618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:56.416350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 501/1023] 10.222.0.33 () {56 vars in 1105 bytes} [Sun Sep 11 05:17:56 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:56.419088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22044181-4c45-4679-b9e6-f4381b024ab0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:56.518481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22044181-4c45-4679-b9e6-f4381b024ab0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:56.525482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:56.554562 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bd59a-48e5-4745-8dd6-e9426b832ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:56.555184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bd59a-48e5-4745-8dd6-e9426b832ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:56.555466 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bd59a-48e5-4745-8dd6-e9426b832ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:56.555793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bd59a-48e5-4745-8dd6-e9426b832ef0 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:56.556457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bd59a-48e5-4745-8dd6-e9426b832ef0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:56.556827 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bd59a-48e5-4745-8dd6-e9426b832ef0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:56.558195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bd59a-48e5-4745-8dd6-e9426b832ef0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:56.567312 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 523/1024] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:56.568897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e67501-428b-47df-9413-976adb5f149d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:56.673407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e67501-428b-47df-9413-976adb5f149d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:56.681057 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:56.709900 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeabba-bab0-4ebc-921f-696982b779c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:56.710513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeabba-bab0-4ebc-921f-696982b779c2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:56.710786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeabba-bab0-4ebc-921f-696982b779c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:56.711051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeabba-bab0-4ebc-921f-696982b779c2 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:56.711672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eeabba-bab0-4ebc-921f-696982b779c2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:56.711970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eeabba-bab0-4ebc-921f-696982b779c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:56.713352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eeabba-bab0-4ebc-921f-696982b779c2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:56.718340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 502/1025] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:56.724509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bd59a-48e5-4745-8dd6-e9426b832ef0 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:56.838744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bd59a-48e5-4745-8dd6-e9426b832ef0 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:56.873584 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb5835-3cd5-450b-9723-24aee90b4904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:56.874207 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb5835-3cd5-450b-9723-24aee90b4904 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:56.874488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb5835-3cd5-450b-9723-24aee90b4904 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:56.874761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb5835-3cd5-450b-9723-24aee90b4904 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:56.876235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb5835-3cd5-450b-9723-24aee90b4904 None admin] RBAC: Adding query filter params (interface=public, service_id=7b2e4e6a7a5748b3bfc2c90347594954, region_id=RegionOne) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:56.876456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb5835-3cd5-450b-9723-24aee90b4904 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:56.877938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb5835-3cd5-450b-9723-24aee90b4904 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:56.888225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 524/1026] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 05:17:56 2022] GET /identity/v3/endpoints?service_id=7b2e4e6a7a5748b3bfc2c90347594954&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:17:56.889172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeabba-bab0-4ebc-921f-696982b779c2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:56.994881 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeabba-bab0-4ebc-921f-696982b779c2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:57.031422 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa497785-6dfb-458d-8803-9d0645724783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:57.032183 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa497785-6dfb-458d-8803-9d0645724783 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:57.032472 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa497785-6dfb-458d-8803-9d0645724783 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:57.032745 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa497785-6dfb-458d-8803-9d0645724783 None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:57.033349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa497785-6dfb-458d-8803-9d0645724783 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:57.034722 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa497785-6dfb-458d-8803-9d0645724783 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:57.041774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 503/1027] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 05:17:56 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:17:57.230930 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb30b0-914f-46d1-8714-3c84505fa2f7 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:57.231211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb30b0-914f-46d1-8714-3c84505fa2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:57.231483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb30b0-914f-46d1-8714-3c84505fa2f7 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:57.233179 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 525/1028] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:57.239132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a22cb-faef-4147-ac12-8c49689741c1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:57.239418 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a22cb-faef-4147-ac12-8c49689741c1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:57.239690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a22cb-faef-4147-ac12-8c49689741c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:57.259701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-014a22cb-faef-4147-ac12-8c49689741c1 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:57.260990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-014a22cb-faef-4147-ac12-8c49689741c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:57.289267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:57.332675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 504/1029] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:17:57.604663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb30b0-914f-46d1-8714-3c84505fa2f7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:57.629997 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5770cce-d78c-4f9e-b009-55ad096654e9 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:57.630329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5770cce-d78c-4f9e-b009-55ad096654e9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:57.630552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5770cce-d78c-4f9e-b009-55ad096654e9 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:57.632172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 505/1030] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:17:57.636968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759209f3-72ba-4b8b-97a1-b63f9a872385 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:57.637262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759209f3-72ba-4b8b-97a1-b63f9a872385 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:57.637533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759209f3-72ba-4b8b-97a1-b63f9a872385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:57.649120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:57.656233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-759209f3-72ba-4b8b-97a1-b63f9a872385 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:57.657766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-759209f3-72ba-4b8b-97a1-b63f9a872385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:57.721389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb30b0-914f-46d1-8714-3c84505fa2f7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:57.727476 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 506/1031] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:57.757328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abe4b8-aba0-4eee-a71f-acf37ddba851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:57.757959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abe4b8-aba0-4eee-a71f-acf37ddba851 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:57.758239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abe4b8-aba0-4eee-a71f-acf37ddba851 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:57.758506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abe4b8-aba0-4eee-a71f-acf37ddba851 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:57.759604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abe4b8-aba0-4eee-a71f-acf37ddba851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:57.761125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abe4b8-aba0-4eee-a71f-acf37ddba851 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:57.764025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 526/1032] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:57.769205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759209f3-72ba-4b8b-97a1-b63f9a872385 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:57.875124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:57.882842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759209f3-72ba-4b8b-97a1-b63f9a872385 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:57.918615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26513eaa-3a9e-41a7-879d-3dbf95861f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:57.919241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26513eaa-3a9e-41a7-879d-3dbf95861f68 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:57.919491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26513eaa-3a9e-41a7-879d-3dbf95861f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:57.919765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26513eaa-3a9e-41a7-879d-3dbf95861f68 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:57.920458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26513eaa-3a9e-41a7-879d-3dbf95861f68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:57.921898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26513eaa-3a9e-41a7-879d-3dbf95861f68 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:57.945043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 507/1033] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:17:57 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:17:58.001833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abe4b8-aba0-4eee-a71f-acf37ddba851 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:58.108422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abe4b8-aba0-4eee-a71f-acf37ddba851 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:58.110057 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:58.148814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ae7ad-e8e7-46b7-916e-c6348b63272b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:58.149363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ae7ad-e8e7-46b7-916e-c6348b63272b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:58.149666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ae7ad-e8e7-46b7-916e-c6348b63272b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:58.149937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ae7ad-e8e7-46b7-916e-c6348b63272b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:58.160438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ae7ad-e8e7-46b7-916e-c6348b63272b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-3f1ae7ad-e8e7-46b7-916e-c6348b63272b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:17:58.165270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:58.166206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:58.167070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:58.167070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:58.167070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:58.167070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:58.167070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:17:58.167070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:17:58.167070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:17:58.167070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:17:58.167070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:17:58.167070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:17:58.167070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 05:17:58.167070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:58.167070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 527/1034] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 11 05:17:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:58.172509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26513eaa-3a9e-41a7-879d-3dbf95861f68 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:58.284294 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26513eaa-3a9e-41a7-879d-3dbf95861f68 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:58.318675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cc757-c32b-4475-9d05-059942270df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:58.319289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cc757-c32b-4475-9d05-059942270df3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:58.319569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cc757-c32b-4475-9d05-059942270df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:58.319842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cc757-c32b-4475-9d05-059942270df3 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:58.320536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cc757-c32b-4475-9d05-059942270df3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:58.320842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cc757-c32b-4475-9d05-059942270df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:58.322217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cc757-c32b-4475-9d05-059942270df3 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:58.334237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 508/1035] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 05:17:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:58.337824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ae7ad-e8e7-46b7-916e-c6348b63272b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:58.438818 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ae7ad-e8e7-46b7-916e-c6348b63272b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:58.473424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aeea3-ec45-449c-b3c7-1bb73d2051e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:58.474057 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aeea3-ec45-449c-b3c7-1bb73d2051e4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:58.474338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aeea3-ec45-449c-b3c7-1bb73d2051e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:58.474610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aeea3-ec45-449c-b3c7-1bb73d2051e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:58.485375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41aeea3-ec45-449c-b3c7-1bb73d2051e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:58.486814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41aeea3-ec45-449c-b3c7-1bb73d2051e4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d41aeea3-ec45-449c-b3c7-1bb73d2051e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:58.501804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:58.502769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:58.503681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:58.503681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:58.503681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:58.503681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 528/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:17:58.510235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cc757-c32b-4475-9d05-059942270df3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:58.526394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:58.616582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cc757-c32b-4475-9d05-059942270df3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:58.650738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb42d8f-adb9-43cb-9d55-92133e4e0674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:58.651345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb42d8f-adb9-43cb-9d55-92133e4e0674 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:58.651624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb42d8f-adb9-43cb-9d55-92133e4e0674 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:58.651894 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb42d8f-adb9-43cb-9d55-92133e4e0674 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:58.652580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb42d8f-adb9-43cb-9d55-92133e4e0674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:58.652941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb42d8f-adb9-43cb-9d55-92133e4e0674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:58.654271 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb42d8f-adb9-43cb-9d55-92133e4e0674 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:58.666677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 509/1037] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:58.672589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aeea3-ec45-449c-b3c7-1bb73d2051e4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:58.689196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:58.775554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aeea3-ec45-449c-b3c7-1bb73d2051e4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:58.817416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9fee0-8ae7-4bec-9b60-818c62727875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:58.818052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9fee0-8ae7-4bec-9b60-818c62727875 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:58.818332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9fee0-8ae7-4bec-9b60-818c62727875 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:58.818601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9fee0-8ae7-4bec-9b60-818c62727875 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:58.826163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9fee0-8ae7-4bec-9b60-818c62727875 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:58.827597 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9fee0-8ae7-4bec-9b60-818c62727875 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-c6b9fee0-8ae7-4bec-9b60-818c62727875 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:17:58.838544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:58.839497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:58.840500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:58.840500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:58.840500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:17:58.840500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:17:58.840500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:17:58.840500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:17:58.840500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:17:58.840500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:17:58.840500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:17:58.840500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:58.844853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 529/1038] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 05:17:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:58.846230 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb42d8f-adb9-43cb-9d55-92133e4e0674 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:58.921091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:58.953685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb42d8f-adb9-43cb-9d55-92133e4e0674 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:58.988673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971b53d-f96e-4195-8cc7-d2a0bd61c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:58.989273 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971b53d-f96e-4195-8cc7-d2a0bd61c541 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:58.989552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971b53d-f96e-4195-8cc7-d2a0bd61c541 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:58.989823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971b53d-f96e-4195-8cc7-d2a0bd61c541 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:58.990542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971b53d-f96e-4195-8cc7-d2a0bd61c541 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:58.990844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971b53d-f96e-4195-8cc7-d2a0bd61c541 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:58.992298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e971b53d-f96e-4195-8cc7-d2a0bd61c541 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:59.015199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 510/1039] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 05:17:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:59.017502 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9fee0-8ae7-4bec-9b60-818c62727875 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:59.092262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:59.118782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9fee0-8ae7-4bec-9b60-818c62727875 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:59.154729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e461600-5d73-4126-87bb-f16fbbc7de0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:59.155359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e461600-5d73-4126-87bb-f16fbbc7de0b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:59.155639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e461600-5d73-4126-87bb-f16fbbc7de0b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:59.155909 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e461600-5d73-4126-87bb-f16fbbc7de0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:59.226359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e461600-5d73-4126-87bb-f16fbbc7de0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:59.227982 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e461600-5d73-4126-87bb-f16fbbc7de0b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:59.262573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75ffee-992e-4ef2-a464-bf1fee5aa004 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:59.262874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75ffee-992e-4ef2-a464-bf1fee5aa004 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:59.263124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75ffee-992e-4ef2-a464-bf1fee5aa004 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:59.264629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 511/1040] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:17:59.269539 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05931b-31b7-4fdc-9f46-b12265944f0c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:59.269826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05931b-31b7-4fdc-9f46-b12265944f0c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:59.270099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05931b-31b7-4fdc-9f46-b12265944f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:59.287636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-fe05931b-31b7-4fdc-9f46-b12265944f0c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:17:59.288871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fe05931b-31b7-4fdc-9f46-b12265944f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:17:59.304854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:59.340506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 530/1041] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:17:59 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:17:59.342207 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e461600-5d73-4126-87bb-f16fbbc7de0b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:59.362205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 512/1042] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:17:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:17:59.443259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e461600-5d73-4126-87bb-f16fbbc7de0b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:59.477205 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f1432-3f0d-4cfe-99a3-28efad363c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:59.478007 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f1432-3f0d-4cfe-99a3-28efad363c4e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:59.478427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f1432-3f0d-4cfe-99a3-28efad363c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:59.478840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f1432-3f0d-4cfe-99a3-28efad363c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:59.488868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f1432-3f0d-4cfe-99a3-28efad363c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:59.490441 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f1432-3f0d-4cfe-99a3-28efad363c4e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5d3f1432-3f0d-4cfe-99a3-28efad363c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:17:59.505785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:59.506904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:17:59.508030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:17:59.508030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:17:59.508030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:59.509364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 531/1043] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:59.515232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05931b-31b7-4fdc-9f46-b12265944f0c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:59.581828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:59.629287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05931b-31b7-4fdc-9f46-b12265944f0c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:59.636195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f1432-3f0d-4cfe-99a3-28efad363c4e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:59.665535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:59.665748 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7e62f-f5c3-4788-b3fd-1dae36b18d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:59.666333 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7e62f-f5c3-4788-b3fd-1dae36b18d71 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:59.666610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7e62f-f5c3-4788-b3fd-1dae36b18d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:59.667006 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7e62f-f5c3-4788-b3fd-1dae36b18d71 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:59.667986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7e62f-f5c3-4788-b3fd-1dae36b18d71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:59.668356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7e62f-f5c3-4788-b3fd-1dae36b18d71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:59.669779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7e62f-f5c3-4788-b3fd-1dae36b18d71 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:59.688699 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 513/1044] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:59.689848 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7e62f-f5c3-4788-b3fd-1dae36b18d71 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:59.750031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f1432-3f0d-4cfe-99a3-28efad363c4e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:59.789632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50440058-fcbb-4b11-94e4-2a3ebbf1552b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:59.790559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50440058-fcbb-4b11-94e4-2a3ebbf1552b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:59.790851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50440058-fcbb-4b11-94e4-2a3ebbf1552b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:59.791126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50440058-fcbb-4b11-94e4-2a3ebbf1552b None admin] PATH_INFO: `/v3/services/image` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:59.791717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50440058-fcbb-4b11-94e4-2a3ebbf1552b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:59.793257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50440058-fcbb-4b11-94e4-2a3ebbf1552b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:59.801809 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7e62f-f5c3-4788-b3fd-1dae36b18d71 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-50440058-fcbb-4b11-94e4-2a3ebbf1552b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:59.803953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:59.805162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 05:17:59.806288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:17:59.809962 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 532/1045] 10.222.0.33 () {56 vars in 1105 bytes} [Sun Sep 11 05:17:59 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:59.810792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50440058-fcbb-4b11-94e4-2a3ebbf1552b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:59.829427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:59.846314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e4e3c-734f-4eda-a61e-08f54a4804bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:59.847072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e4e3c-734f-4eda-a61e-08f54a4804bd None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:59.847351 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e4e3c-734f-4eda-a61e-08f54a4804bd None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:59.847625 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e4e3c-734f-4eda-a61e-08f54a4804bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:59.854932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e4e3c-734f-4eda-a61e-08f54a4804bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:59.856514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e4e3c-734f-4eda-a61e-08f54a4804bd None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-669e4e3c-734f-4eda-a61e-08f54a4804bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:17:59.869516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:17:59.870441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:17:59.871363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:17:59.871363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:17:59.871363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:17:59.871363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:17:59.871363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:17:59.871363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:17:59.871363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:17:59.871363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:17:59.871363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 514/1046] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:17:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:17:59.877173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e4e3c-734f-4eda-a61e-08f54a4804bd None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:59.905614 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:17:59.917199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50440058-fcbb-4b11-94e4-2a3ebbf1552b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:17:59.953610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910fe91-36ce-4f7f-bedb-ab91f954a24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:17:59.954221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910fe91-36ce-4f7f-bedb-ab91f954a24c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:17:59.954506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910fe91-36ce-4f7f-bedb-ab91f954a24c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:17:59.954776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910fe91-36ce-4f7f-bedb-ab91f954a24c None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:17:59.955411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910fe91-36ce-4f7f-bedb-ab91f954a24c None admin] RBAC: Adding query filter params (name=image) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:17:59.955720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910fe91-36ce-4f7f-bedb-ab91f954a24c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:17:59.957117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910fe91-36ce-4f7f-bedb-ab91f954a24c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:17:59.962243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 533/1047] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:17:59.967763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910fe91-36ce-4f7f-bedb-ab91f954a24c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:17:59.982007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e4e3c-734f-4eda-a61e-08f54a4804bd None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:00.017500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e0f86-229f-4e27-b155-2069f2f56bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:00.018025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e0f86-229f-4e27-b155-2069f2f56bba None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:00.018310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e0f86-229f-4e27-b155-2069f2f56bba None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:00.018586 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e0f86-229f-4e27-b155-2069f2f56bba None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:00.019282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e0f86-229f-4e27-b155-2069f2f56bba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:00.019588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e0f86-229f-4e27-b155-2069f2f56bba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:00.021061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e0f86-229f-4e27-b155-2069f2f56bba None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:00.038147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 515/1048] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:17:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:00.039575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e0f86-229f-4e27-b155-2069f2f56bba None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:00.081013 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910fe91-36ce-4f7f-bedb-ab91f954a24c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:00.087263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:00.121471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a495a-9776-409e-a052-6b9e2ab80a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:00.122079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a495a-9776-409e-a052-6b9e2ab80a6e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:00.122360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a495a-9776-409e-a052-6b9e2ab80a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:00.122635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a495a-9776-409e-a052-6b9e2ab80a6e None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:00.123262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a495a-9776-409e-a052-6b9e2ab80a6e None admin] RBAC: Adding query filter params (type=image) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:00.123682 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a495a-9776-409e-a052-6b9e2ab80a6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:00.125080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a495a-9776-409e-a052-6b9e2ab80a6e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:00.135497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 534/1049] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:17:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:00.136500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a495a-9776-409e-a052-6b9e2ab80a6e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:00.155168 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e0f86-229f-4e27-b155-2069f2f56bba None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:00.157739 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:00.192066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e9358-7b34-4746-90c5-90c554392bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:00.192684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e9358-7b34-4746-90c5-90c554392bae None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:00.192966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e9358-7b34-4746-90c5-90c554392bae None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:00.193239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e9358-7b34-4746-90c5-90c554392bae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:00.193996 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e9358-7b34-4746-90c5-90c554392bae None admin] RBAC: Adding query filter params (role.id=a1192a1e2adb45d490e8d553aa87c6a6, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=3bd5a9a09f034c05815c631a44d1cdd9) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:00.194307 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e9358-7b34-4746-90c5-90c554392bae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:00.195741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e9358-7b34-4746-90c5-90c554392bae None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:00.201141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 516/1050] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:18:00 2022] GET /identity/v3/role_assignments?user.id=3bd5a9a09f034c05815c631a44d1cdd9&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=a1192a1e2adb45d490e8d553aa87c6a6 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:00.202278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7da039-4b9f-45d7-8d38-f225bafb9367 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:00.202633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7da039-4b9f-45d7-8d38-f225bafb9367 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:00.202991 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7da039-4b9f-45d7-8d38-f225bafb9367 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:00.204980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 517/1051] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:00.211851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8d646-8cc4-48b1-885f-c30209b7e4b9 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:00.212500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8d646-8cc4-48b1-885f-c30209b7e4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:00.212500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8d646-8cc4-48b1-885f-c30209b7e4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:00.231679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8ac8d646-8cc4-48b1-885f-c30209b7e4b9 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:00.232968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8ac8d646-8cc4-48b1-885f-c30209b7e4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:00.260427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a495a-9776-409e-a052-6b9e2ab80a6e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:00.301928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68201c75-8e25-49c2-8c27-2421531d8e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:00.302464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68201c75-8e25-49c2-8c27-2421531d8e72 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:00.302749 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68201c75-8e25-49c2-8c27-2421531d8e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:00.303037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68201c75-8e25-49c2-8c27-2421531d8e72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:00.303956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68201c75-8e25-49c2-8c27-2421531d8e72 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:00.305137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68201c75-8e25-49c2-8c27-2421531d8e72 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:00.308805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 518/1052] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:18:00.327288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 535/1053] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:18:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:18:00.593369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8d646-8cc4-48b1-885f-c30209b7e4b9 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:00.630014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:00.702345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8d646-8cc4-48b1-885f-c30209b7e4b9 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:00.744826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf3158-6ca4-4836-a8f7-5478ece36e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:00.745561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf3158-6ca4-4836-a8f7-5478ece36e65 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:00.745843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf3158-6ca4-4836-a8f7-5478ece36e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:00.746110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf3158-6ca4-4836-a8f7-5478ece36e65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:00.756823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf3158-6ca4-4836-a8f7-5478ece36e65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-19bf3158-6ca4-4836-a8f7-5478ece36e65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:18:00.761574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:00.762532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:00.763388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:00.763388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:00.763388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:00.763388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:00.763388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:18:00.763388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:18:00.763388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:18:00.763388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:18:00.763388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:18:00.763388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:18:00.763388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:00.763388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:18:00.768091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 519/1054] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:18:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:00.769757 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68201c75-8e25-49c2-8c27-2421531d8e72 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:00.855233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:00.883855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68201c75-8e25-49c2-8c27-2421531d8e72 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:00.919143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d2d9f-b4b1-4645-b442-1120f5fc1471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:00.919785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d2d9f-b4b1-4645-b442-1120f5fc1471 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:00.920087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d2d9f-b4b1-4645-b442-1120f5fc1471 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:00.920358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d2d9f-b4b1-4645-b442-1120f5fc1471 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:00.921020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d2d9f-b4b1-4645-b442-1120f5fc1471 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:00.921318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d2d9f-b4b1-4645-b442-1120f5fc1471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:00.923113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d2d9f-b4b1-4645-b442-1120f5fc1471 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:00.932500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 536/1055] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:18:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:00.938880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf3158-6ca4-4836-a8f7-5478ece36e65 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:01.031353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:01.043697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf3158-6ca4-4836-a8f7-5478ece36e65 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:01.081755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8f916-c524-4898-bd80-ef94fe71e564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:01.082402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8f916-c524-4898-bd80-ef94fe71e564 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:01.082678 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8f916-c524-4898-bd80-ef94fe71e564 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:01.082945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8f916-c524-4898-bd80-ef94fe71e564 None admin] PATH_INFO: `/v3/users/9025165ba2ca4fb69c65ca92d178890e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:01.098326 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b8f916-c524-4898-bd80-ef94fe71e564 None admin] RBAC: Authorizing `identity:get_user(user_id=9025165ba2ca4fb69c65ca92d178890e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:01.099777 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b8f916-c524-4898-bd80-ef94fe71e564 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:01.106147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 520/1056] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:00 2022] GET /identity/v3/users/9025165ba2ca4fb69c65ca92d178890e => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:01.107277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d2d9f-b4b1-4645-b442-1120f5fc1471 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:01.208168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d2d9f-b4b1-4645-b442-1120f5fc1471 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:01.244765 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303b592-fbf6-4760-a8f6-4bfd8f362ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:01.245377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303b592-fbf6-4760-a8f6-4bfd8f362ebd None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:01.245650 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303b592-fbf6-4760-a8f6-4bfd8f362ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:01.245920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303b592-fbf6-4760-a8f6-4bfd8f362ebd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:01.249309 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:01.316834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303b592-fbf6-4760-a8f6-4bfd8f362ebd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:01.318265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303b592-fbf6-4760-a8f6-4bfd8f362ebd None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:01.421408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 537/1057] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:18:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 05:18:01.423501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8f916-c524-4898-bd80-ef94fe71e564 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:01.524622 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8f916-c524-4898-bd80-ef94fe71e564 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:01.560034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d732f-d30d-461a-a924-a93b280869fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:01.560673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d732f-d30d-461a-a924-a93b280869fe None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:01.560953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d732f-d30d-461a-a924-a93b280869fe None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:01.561436 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d732f-d30d-461a-a924-a93b280869fe None admin] PATH_INFO: `/v3/projects/0664a432c1024a8a82884e5b3ae76cb6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:01.562714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d732f-d30d-461a-a924-a93b280869fe None admin] RBAC: Authorizing `identity:get_project(project_id=0664a432c1024a8a82884e5b3ae76cb6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:01.564681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d732f-d30d-461a-a924-a93b280869fe None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:01.566464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 521/1058] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:18:01 2022] GET /identity/v3/projects/0664a432c1024a8a82884e5b3ae76cb6 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:01.572386 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303b592-fbf6-4760-a8f6-4bfd8f362ebd None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:01.608753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:01.677548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303b592-fbf6-4760-a8f6-4bfd8f362ebd None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:01.711126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c13c17-9b54-41d6-92b0-0fd9dbec8bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:01.711731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c13c17-9b54-41d6-92b0-0fd9dbec8bad None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:01.712030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c13c17-9b54-41d6-92b0-0fd9dbec8bad None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:01.712374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c13c17-9b54-41d6-92b0-0fd9dbec8bad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:01.713103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c13c17-9b54-41d6-92b0-0fd9dbec8bad None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=0664a432c1024a8a82884e5b3ae76cb6, user.id=9025165ba2ca4fb69c65ca92d178890e) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:01.713414 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c13c17-9b54-41d6-92b0-0fd9dbec8bad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:01.714833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c13c17-9b54-41d6-92b0-0fd9dbec8bad None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:01.721494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 538/1059] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:18:01 2022] GET /identity/v3/role_assignments?user.id=9025165ba2ca4fb69c65ca92d178890e&scope.project.id=0664a432c1024a8a82884e5b3ae76cb6&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:02.358824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657f6e1-f1a8-4e0f-902a-16e85b11bec7 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:02.360259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657f6e1-f1a8-4e0f-902a-16e85b11bec7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:02.360259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657f6e1-f1a8-4e0f-902a-16e85b11bec7 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:02.361450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 522/1060] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:02.366553 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b5db4-57ef-4c16-b374-1551ed6a41a7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:02.367001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b5db4-57ef-4c16-b374-1551ed6a41a7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:02.367096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b5db4-57ef-4c16-b374-1551ed6a41a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:02.386249 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-902b5db4-57ef-4c16-b374-1551ed6a41a7 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:02.387470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-902b5db4-57ef-4c16-b374-1551ed6a41a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:02.459488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 539/1061] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:02.518109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4c661-7547-4861-ac30-237c56a0900d None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:02.518451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4c661-7547-4861-ac30-237c56a0900d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:02.518655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4c661-7547-4861-ac30-237c56a0900d None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:02.520049 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 523/1062] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:18:02.525196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f72bc2-a10a-4686-8b4e-905bfe6e62f9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:02.525481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f72bc2-a10a-4686-8b4e-905bfe6e62f9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:02.525752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f72bc2-a10a-4686-8b4e-905bfe6e62f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:02.543393 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e3f72bc2-a10a-4686-8b4e-905bfe6e62f9 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:02.544584 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e3f72bc2-a10a-4686-8b4e-905bfe6e62f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:02.568142 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:02.614568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 540/1063] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:02.724552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4c661-7547-4861-ac30-237c56a0900d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:02.742604 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:02.837758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4c661-7547-4861-ac30-237c56a0900d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:02.872322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b54d96-4e0b-4ce2-b098-11af18395782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:02.872873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b54d96-4e0b-4ce2-b098-11af18395782 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:02.873150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b54d96-4e0b-4ce2-b098-11af18395782 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:02.873423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b54d96-4e0b-4ce2-b098-11af18395782 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:02.882304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f72bc2-a10a-4686-8b4e-905bfe6e62f9 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:02.884969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b54d96-4e0b-4ce2-b098-11af18395782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-f3b54d96-4e0b-4ce2-b098-11af18395782 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:18:02.889755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:02.891295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:02.892647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:02.892647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:02.892647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:02.892647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:02.892647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:18:02.892647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:18:02.892647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:18:02.892647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:18:02.892647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:18:02.892647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:18:02.892647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 05:18:02.892647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:18:02.892647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 524/1064] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 11 05:18:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:02.897129 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b54d96-4e0b-4ce2-b098-11af18395782 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:02.982078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:02.992417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f72bc2-a10a-4686-8b4e-905bfe6e62f9 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:03.006152 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b54d96-4e0b-4ce2-b098-11af18395782 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:03.028436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf6024-ae4b-4c05-8825-7c3754cb2226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:03.029030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf6024-ae4b-4c05-8825-7c3754cb2226 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:03.029309 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf6024-ae4b-4c05-8825-7c3754cb2226 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:03.029579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf6024-ae4b-4c05-8825-7c3754cb2226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:03.045475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fd192-1085-4ba8-9021-a2865fa4ef64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:03.046227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fd192-1085-4ba8-9021-a2865fa4ef64 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:03.046632 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fd192-1085-4ba8-9021-a2865fa4ef64 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:03.046910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fd192-1085-4ba8-9021-a2865fa4ef64 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:03.048560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fd192-1085-4ba8-9021-a2865fa4ef64 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:03.048874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fd192-1085-4ba8-9021-a2865fa4ef64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:03.050660 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fd192-1085-4ba8-9021-a2865fa4ef64 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:03.058667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:03.068919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 525/1065] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 05:18:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:03.069668 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fd192-1085-4ba8-9021-a2865fa4ef64 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:03.100748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf6024-ae4b-4c05-8825-7c3754cb2226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:03.102184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf6024-ae4b-4c05-8825-7c3754cb2226 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:03.169243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fd192-1085-4ba8-9021-a2865fa4ef64 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:03.207714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b79904-a69a-4121-8456-bd49302e8f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:03.208364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b79904-a69a-4121-8456-bd49302e8f5c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:03.208639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b79904-a69a-4121-8456-bd49302e8f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:03.208908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b79904-a69a-4121-8456-bd49302e8f5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:03.212068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 541/1066] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:18:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 05:18:03.215562 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf6024-ae4b-4c05-8825-7c3754cb2226 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:03.218492 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:03.222806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b79904-a69a-4121-8456-bd49302e8f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:03.224527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b79904-a69a-4121-8456-bd49302e8f5c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-f5b79904-a69a-4121-8456-bd49302e8f5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:18:03.241412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:03.242576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:18:03.243671 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:18:03.243671 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:18:03.243671 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:18:03.245060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 526/1067] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:03.250673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b79904-a69a-4121-8456-bd49302e8f5c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:03.287587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:03.333929 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf6024-ae4b-4c05-8825-7c3754cb2226 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:03.356327 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b79904-a69a-4121-8456-bd49302e8f5c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:03.371849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ffa74-dfd7-4175-9665-8fc48db2a5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:03.372506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ffa74-dfd7-4175-9665-8fc48db2a5db None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:03.372781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ffa74-dfd7-4175-9665-8fc48db2a5db None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:03.373048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ffa74-dfd7-4175-9665-8fc48db2a5db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:03.383911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ffa74-dfd7-4175-9665-8fc48db2a5db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:03.385370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ffa74-dfd7-4175-9665-8fc48db2a5db None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:03.392846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597b0f9-f687-4e9e-a4af-0abf509d7aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:03.393451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597b0f9-f687-4e9e-a4af-0abf509d7aeb None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:03.393731 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597b0f9-f687-4e9e-a4af-0abf509d7aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:03.394003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597b0f9-f687-4e9e-a4af-0abf509d7aeb None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:03.394644 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597b0f9-f687-4e9e-a4af-0abf509d7aeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:03.394954 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597b0f9-f687-4e9e-a4af-0abf509d7aeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:03.396342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597b0f9-f687-4e9e-a4af-0abf509d7aeb None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-6c6ffa74-dfd7-4175-9665-8fc48db2a5db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:18:03.401474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:03.402629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:18:03.403725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:18:03.403725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:18:03.403725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:18:03.403725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 542/1068] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:18:03.414756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ffa74-dfd7-4175-9665-8fc48db2a5db None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:03.414924 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 527/1069] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:03.415809 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597b0f9-f687-4e9e-a4af-0abf509d7aeb None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:03.466613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:03.517808 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597b0f9-f687-4e9e-a4af-0abf509d7aeb None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:03.522601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ffa74-dfd7-4175-9665-8fc48db2a5db None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:03.537875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:03.554717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b77408-763e-4165-9cbb-b2be153d5e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:03.555442 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b77408-763e-4165-9cbb-b2be153d5e4d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:03.555797 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b77408-763e-4165-9cbb-b2be153d5e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:03.556172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b77408-763e-4165-9cbb-b2be153d5e4d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:03.562386 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7ae18-7a9c-4760-be05-da001755bbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:03.563258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7ae18-7a9c-4760-be05-da001755bbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:03.563642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7ae18-7a9c-4760-be05-da001755bbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:03.563746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b77408-763e-4165-9cbb-b2be153d5e4d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:03.564060 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7ae18-7a9c-4760-be05-da001755bbf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:03.564767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7ae18-7a9c-4760-be05-da001755bbf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:03.565137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7ae18-7a9c-4760-be05-da001755bbf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:03.565506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b77408-763e-4165-9cbb-b2be153d5e4d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:03.566796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7ae18-7a9c-4760-be05-da001755bbf7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-90b77408-763e-4165-9cbb-b2be153d5e4d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:18:03.579524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:03.580820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:03.582030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:03.582030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:03.582030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:18:03.582030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:18:03.582030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:18:03.582030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:18:03.582030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:18:03.582030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:18:03.582030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:18:03.582030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:18:03.582674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 528/1070] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 05:18:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:03.588085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 543/1071] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:03.589193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b77408-763e-4165-9cbb-b2be153d5e4d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:03.703824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b77408-763e-4165-9cbb-b2be153d5e4d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:03.715647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:03.743967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3b585-bde5-44b3-87d9-0c626d41583b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:03.744873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3b585-bde5-44b3-87d9-0c626d41583b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:03.745306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3b585-bde5-44b3-87d9-0c626d41583b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:03.745734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3b585-bde5-44b3-87d9-0c626d41583b None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:03.746637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3b585-bde5-44b3-87d9-0c626d41583b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:03.747100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3b585-bde5-44b3-87d9-0c626d41583b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:03.748754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3b585-bde5-44b3-87d9-0c626d41583b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:03.771658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 529/1072] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 05:18:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:03.778123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7ae18-7a9c-4760-be05-da001755bbf7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:03.882919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965be59-6560-4c45-8e4e-0a3c4b9ac994 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:03.883345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965be59-6560-4c45-8e4e-0a3c4b9ac994 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:03.883766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965be59-6560-4c45-8e4e-0a3c4b9ac994 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:03.885249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 530/1073] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:18:03.892383 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3865e4-9715-411a-b0a2-74d8b61ae98c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:03.892675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3865e4-9715-411a-b0a2-74d8b61ae98c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:03.892957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3865e4-9715-411a-b0a2-74d8b61ae98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:03.894926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7ae18-7a9c-4760-be05-da001755bbf7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:03.912614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5a3865e4-9715-411a-b0a2-74d8b61ae98c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:03.913815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5a3865e4-9715-411a-b0a2-74d8b61ae98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:03.933021 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a6a66-dd70-4eb5-b8f8-ce778573fa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:03.933659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a6a66-dd70-4eb5-b8f8-ce778573fa8d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:03.933942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a6a66-dd70-4eb5-b8f8-ce778573fa8d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:03.934216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a6a66-dd70-4eb5-b8f8-ce778573fa8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:03.945364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a6a66-dd70-4eb5-b8f8-ce778573fa8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:03.946835 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a6a66-dd70-4eb5-b8f8-ce778573fa8d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-816a6a66-dd70-4eb5-b8f8-ce778573fa8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:18:03.961408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:03.962569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:18:03.963667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:18:03.963667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:18:03.963667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:18:03.963667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 544/1074] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:03.968147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a6a66-dd70-4eb5-b8f8-ce778573fa8d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:03.985135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 531/1075] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:18:03.994618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:04.066782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a6a66-dd70-4eb5-b8f8-ce778573fa8d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:04.098884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6e69e-4a91-47bb-b238-369132b3dfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:04.099481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6e69e-4a91-47bb-b238-369132b3dfb3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:04.099760 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6e69e-4a91-47bb-b238-369132b3dfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:04.100064 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6e69e-4a91-47bb-b238-369132b3dfb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:04.100699 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6e69e-4a91-47bb-b238-369132b3dfb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:04.101002 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6e69e-4a91-47bb-b238-369132b3dfb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:04.102345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6e69e-4a91-47bb-b238-369132b3dfb3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:04.113655 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 545/1076] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:04.119228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3865e4-9715-411a-b0a2-74d8b61ae98c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:04.213919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:04.217724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3865e4-9715-411a-b0a2-74d8b61ae98c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:04.251888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbf6af-0d7e-4725-9948-97a15b3eb285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:04.252605 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbf6af-0d7e-4725-9948-97a15b3eb285 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:04.252884 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbf6af-0d7e-4725-9948-97a15b3eb285 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:04.253155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbf6af-0d7e-4725-9948-97a15b3eb285 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:04.253360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6e69e-4a91-47bb-b238-369132b3dfb3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:04.260451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbf6af-0d7e-4725-9948-97a15b3eb285 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:04.261929 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbf6af-0d7e-4725-9948-97a15b3eb285 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-34bbf6af-0d7e-4725-9948-97a15b3eb285 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:18:04.274536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:04.280518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:04.281922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:18:04.281922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:18:04.281922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:18:04.281922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:18:04.281922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:18:04.281922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:18:04.281922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:18:04.281922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:18:04.281922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 532/1077] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:18:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:04.281922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbf6af-0d7e-4725-9948-97a15b3eb285 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:04.343128 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:04.358687 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6e69e-4a91-47bb-b238-369132b3dfb3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:04.382519 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbf6af-0d7e-4725-9948-97a15b3eb285 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:04.393897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d1880-315c-4703-b3a4-929a89c7279b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:04.394521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d1880-315c-4703-b3a4-929a89c7279b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:04.394796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d1880-315c-4703-b3a4-929a89c7279b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:04.395065 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d1880-315c-4703-b3a4-929a89c7279b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:04.404793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d1880-315c-4703-b3a4-929a89c7279b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-1e6d1880-315c-4703-b3a4-929a89c7279b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:18:04.409861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:04.410951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:04.412341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:04.412341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:04.412341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:04.412341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:04.412341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:18:04.412341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:18:04.412341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:18:04.412341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:18:04.412341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:18:04.412341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:18:04.412341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:04.412341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:18:04.412341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 546/1078] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:18:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:18:04.415972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7355fe-9ad1-4b89-9d3f-6c0bc3bfcc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:04.416598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7355fe-9ad1-4b89-9d3f-6c0bc3bfcc8c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:04.416802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d1880-315c-4703-b3a4-929a89c7279b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:04.416912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7355fe-9ad1-4b89-9d3f-6c0bc3bfcc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:04.417134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7355fe-9ad1-4b89-9d3f-6c0bc3bfcc8c None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:04.417822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7355fe-9ad1-4b89-9d3f-6c0bc3bfcc8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:04.418128 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7355fe-9ad1-4b89-9d3f-6c0bc3bfcc8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:04.419568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7355fe-9ad1-4b89-9d3f-6c0bc3bfcc8c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:04.432966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 533/1079] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:18:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:04.438972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7355fe-9ad1-4b89-9d3f-6c0bc3bfcc8c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:04.457262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:04.517172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d1880-315c-4703-b3a4-929a89c7279b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:04.539366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7355fe-9ad1-4b89-9d3f-6c0bc3bfcc8c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:04.550716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba234f-8f07-480c-95e9-fc9cd7eb46da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:04.551314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba234f-8f07-480c-95e9-fc9cd7eb46da None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:04.551587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba234f-8f07-480c-95e9-fc9cd7eb46da None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:04.551851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba234f-8f07-480c-95e9-fc9cd7eb46da None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:04.552555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba234f-8f07-480c-95e9-fc9cd7eb46da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:04.552849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba234f-8f07-480c-95e9-fc9cd7eb46da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:04.554199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba234f-8f07-480c-95e9-fc9cd7eb46da None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:04.566779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 547/1080] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:18:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:04.567922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba234f-8f07-480c-95e9-fc9cd7eb46da None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:04.575747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:04.579505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfbc73-a664-41cd-a8b2-733c08771f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:04.580194 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfbc73-a664-41cd-a8b2-733c08771f0c None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:04.580476 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfbc73-a664-41cd-a8b2-733c08771f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:04.580746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfbc73-a664-41cd-a8b2-733c08771f0c None admin] PATH_INFO: `/v3/projects/092dd30ff8b541e2a45947504831998a/users/3bd5a9a09f034c05815c631a44d1cdd9/roles/a1192a1e2adb45d490e8d553aa87c6a6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:04.582963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfbc73-a664-41cd-a8b2-733c08771f0c None admin] RBAC: Authorizing `identity:create_grant(project_id=092dd30ff8b541e2a45947504831998a, user_id=3bd5a9a09f034c05815c631a44d1cdd9, role_id=a1192a1e2adb45d490e8d553aa87c6a6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:04.584443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfbc73-a664-41cd-a8b2-733c08771f0c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:04.596920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 534/1081] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:18:04 2022] PUT /identity/v3/projects/092dd30ff8b541e2a45947504831998a/users/3bd5a9a09f034c05815c631a44d1cdd9/roles/a1192a1e2adb45d490e8d553aa87c6a6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:18:04.674546 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba234f-8f07-480c-95e9-fc9cd7eb46da None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:04.709478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69412487-10df-4feb-975e-727afbf2e82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:04.710124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69412487-10df-4feb-975e-727afbf2e82c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:04.710403 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69412487-10df-4feb-975e-727afbf2e82c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:04.710671 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69412487-10df-4feb-975e-727afbf2e82c None admin] PATH_INFO: `/v3/users/9025165ba2ca4fb69c65ca92d178890e` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:04.711837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69412487-10df-4feb-975e-727afbf2e82c None admin] RBAC: Authorizing `identity:get_user(user_id=9025165ba2ca4fb69c65ca92d178890e)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:04.713314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69412487-10df-4feb-975e-727afbf2e82c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:04.714941 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 548/1082] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:04 2022] GET /identity/v3/users/9025165ba2ca4fb69c65ca92d178890e => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:04.721746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfbc73-a664-41cd-a8b2-733c08771f0c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:04.806652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:04.829885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfbc73-a664-41cd-a8b2-733c08771f0c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:04.864150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f28f1-bdb1-4c76-b23c-fd0cec5c5926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:04.864822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f28f1-bdb1-4c76-b23c-fd0cec5c5926 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:04.865101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f28f1-bdb1-4c76-b23c-fd0cec5c5926 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:04.865376 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f28f1-bdb1-4c76-b23c-fd0cec5c5926 None admin] PATH_INFO: `/v3/projects/0664a432c1024a8a82884e5b3ae76cb6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:04.866489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f28f1-bdb1-4c76-b23c-fd0cec5c5926 None admin] RBAC: Authorizing `identity:get_project(project_id=0664a432c1024a8a82884e5b3ae76cb6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:04.867926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f28f1-bdb1-4c76-b23c-fd0cec5c5926 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:04.869880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 535/1083] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:18:04 2022] GET /identity/v3/projects/0664a432c1024a8a82884e5b3ae76cb6 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:04.876332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69412487-10df-4feb-975e-727afbf2e82c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:04.949911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:04.974397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69412487-10df-4feb-975e-727afbf2e82c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:05.007087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc00803-377f-4945-80cf-d6ce638004e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:05.007728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc00803-377f-4945-80cf-d6ce638004e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:05.008022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc00803-377f-4945-80cf-d6ce638004e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:05.008297 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc00803-377f-4945-80cf-d6ce638004e2 None admin] PATH_INFO: `/v3/projects/0664a432c1024a8a82884e5b3ae76cb6/users/9025165ba2ca4fb69c65ca92d178890e/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:05.010153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc00803-377f-4945-80cf-d6ce638004e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0664a432c1024a8a82884e5b3ae76cb6, user_id=9025165ba2ca4fb69c65ca92d178890e, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:05.011568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc00803-377f-4945-80cf-d6ce638004e2 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:05.023790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 549/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:18:04 2022] PUT /identity/v3/projects/0664a432c1024a8a82884e5b3ae76cb6/users/9025165ba2ca4fb69c65ca92d178890e/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:18:06.745730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d7ce9-97bb-4de1-b1bd-fbeafb4c6b5b None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:06.746246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d7ce9-97bb-4de1-b1bd-fbeafb4c6b5b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:06.746325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d7ce9-97bb-4de1-b1bd-fbeafb4c6b5b None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:06.747766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 536/1085] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:06.753288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a14cbc-e53f-48f6-be25-ffbd3fdb3e92 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:06.753719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a14cbc-e53f-48f6-be25-ffbd3fdb3e92 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:06.754162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a14cbc-e53f-48f6-be25-ffbd3fdb3e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:06.772752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-97a14cbc-e53f-48f6-be25-ffbd3fdb3e92 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:06.773944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-97a14cbc-e53f-48f6-be25-ffbd3fdb3e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:06.852481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 550/1086] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:07.126234 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d7ce9-97bb-4de1-b1bd-fbeafb4c6b5b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:07.190618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8a7d3-8736-4e19-81d1-8fca7bf0e005 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:07.190905 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8a7d3-8736-4e19-81d1-8fca7bf0e005 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:07.191182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8a7d3-8736-4e19-81d1-8fca7bf0e005 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:07.192576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 551/1087] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:18:07.197578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d7474-7a9b-496c-8cea-bac54a2d3f84 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:07.197869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d7474-7a9b-496c-8cea-bac54a2d3f84 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:07.198150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d7474-7a9b-496c-8cea-bac54a2d3f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:07.216240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b21d7474-7a9b-496c-8cea-bac54a2d3f84 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:07.217433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b21d7474-7a9b-496c-8cea-bac54a2d3f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:07.230364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d7ce9-97bb-4de1-b1bd-fbeafb4c6b5b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:07.265538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffba1d3-f55f-46ec-9063-d73687b552fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:07.266472 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffba1d3-f55f-46ec-9063-d73687b552fc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:07.266759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffba1d3-f55f-46ec-9063-d73687b552fc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:07.267026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffba1d3-f55f-46ec-9063-d73687b552fc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:07.276920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffba1d3-f55f-46ec-9063-d73687b552fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:18:07.278082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-6ffba1d3-f55f-46ec-9063-d73687b552fc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:18:07.282687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:07.283821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:07.284751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:07.284751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:07.284751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:07.284751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:07.284751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:18:07.284751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:18:07.284751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:18:07.284751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:18:07.284751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:18:07.284751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:18:07.284751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 05:18:07.284751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:18:07.284751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 537/1088] 10.222.0.33 () {56 vars in 1120 bytes} [Sun Sep 11 05:18:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:07.289945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffba1d3-f55f-46ec-9063-d73687b552fc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:07.293092 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 552/1089] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:18:07.299306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:07.394736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffba1d3-f55f-46ec-9063-d73687b552fc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:07.427500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d194b-0a10-4abe-9ea8-f6b3debc8f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:07.428124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d194b-0a10-4abe-9ea8-f6b3debc8f13 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:07.428403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d194b-0a10-4abe-9ea8-f6b3debc8f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:07.428673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d194b-0a10-4abe-9ea8-f6b3debc8f13 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:07.429324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d194b-0a10-4abe-9ea8-f6b3debc8f13 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:07.429621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d194b-0a10-4abe-9ea8-f6b3debc8f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:07.430979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d194b-0a10-4abe-9ea8-f6b3debc8f13 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:07.438781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 538/1090] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 05:18:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:07.444778 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d7474-7a9b-496c-8cea-bac54a2d3f84 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:07.540052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d7474-7a9b-496c-8cea-bac54a2d3f84 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:07.560262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d194b-0a10-4abe-9ea8-f6b3debc8f13 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:07.573489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c05a7-1e91-4561-8b6a-925f315ca948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:07.574114 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c05a7-1e91-4561-8b6a-925f315ca948 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:07.574387 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c05a7-1e91-4561-8b6a-925f315ca948 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:07.574656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c05a7-1e91-4561-8b6a-925f315ca948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:07.585141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c05a7-1e91-4561-8b6a-925f315ca948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:07.586562 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c05a7-1e91-4561-8b6a-925f315ca948 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-001c05a7-1e91-4561-8b6a-925f315ca948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:18:07.602335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:07.603476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:18:07.604431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:18:07.604431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:18:07.604431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:18:07.604431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 553/1091] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:07.609271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c05a7-1e91-4561-8b6a-925f315ca948 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:07.629589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:07.661346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:07.666209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d194b-0a10-4abe-9ea8-f6b3debc8f13 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:07.700025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cd602-eea9-46a0-aa9b-4356857aa4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:07.700646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cd602-eea9-46a0-aa9b-4356857aa4cc None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:07.700925 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cd602-eea9-46a0-aa9b-4356857aa4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:07.701191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cd602-eea9-46a0-aa9b-4356857aa4cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:07.712706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cd602-eea9-46a0-aa9b-4356857aa4cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-997cd602-eea9-46a0-aa9b-4356857aa4cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:18:07.717399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:07.718486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:07.719414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:07.719414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:07.719414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:07.719414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:07.719414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:18:07.719414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:18:07.719414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:18:07.719414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:18:07.719414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:18:07.719414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:18:07.719414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:07.719414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:18:07.719414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 539/1092] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:18:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:18:07.721792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c05a7-1e91-4561-8b6a-925f315ca948 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:07.724366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cd602-eea9-46a0-aa9b-4356857aa4cc None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:07.755845 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76f150-2ba5-47dd-bf96-fff027fbb53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:07.756478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76f150-2ba5-47dd-bf96-fff027fbb53f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:07.756755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76f150-2ba5-47dd-bf96-fff027fbb53f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:07.757024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76f150-2ba5-47dd-bf96-fff027fbb53f None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:07.757663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76f150-2ba5-47dd-bf96-fff027fbb53f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:07.757963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76f150-2ba5-47dd-bf96-fff027fbb53f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:07.759329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76f150-2ba5-47dd-bf96-fff027fbb53f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:07.771518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 554/1093] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:07.777052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76f150-2ba5-47dd-bf96-fff027fbb53f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:07.823785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cd602-eea9-46a0-aa9b-4356857aa4cc None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:07.849255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:07.857367 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f7ea5-e069-49ee-84c0-a88a8643b1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:07.857961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f7ea5-e069-49ee-84c0-a88a8643b1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:07.858238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f7ea5-e069-49ee-84c0-a88a8643b1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:07.858502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f7ea5-e069-49ee-84c0-a88a8643b1b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:07.859153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f7ea5-e069-49ee-84c0-a88a8643b1b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:07.859506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f7ea5-e069-49ee-84c0-a88a8643b1b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:07.861031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f7ea5-e069-49ee-84c0-a88a8643b1b2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:07.869442 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 540/1094] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:18:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:07.875872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f7ea5-e069-49ee-84c0-a88a8643b1b2 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:07.880047 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76f150-2ba5-47dd-bf96-fff027fbb53f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:07.887205 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:07.914635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4930d-3646-48f7-ab0c-f2c5646cdd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:07.915194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4930d-3646-48f7-ab0c-f2c5646cdd0d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:07.915473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4930d-3646-48f7-ab0c-f2c5646cdd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:07.915742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4930d-3646-48f7-ab0c-f2c5646cdd0d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:07.922921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4930d-3646-48f7-ab0c-f2c5646cdd0d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:07.924372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4930d-3646-48f7-ab0c-f2c5646cdd0d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-59b4930d-3646-48f7-ab0c-f2c5646cdd0d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:18:07.935645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:07.936625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:07.938256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:07.938256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:07.938256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:18:07.938256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:18:07.938256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:18:07.938256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:18:07.938256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:18:07.938256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:18:07.938256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 05:18:07.938256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:18:07.938256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 555/1095] 10.222.0.33 () {56 vars in 1093 bytes} [Sun Sep 11 05:18:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:18:07.943112 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4930d-3646-48f7-ab0c-f2c5646cdd0d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:07.973987 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f7ea5-e069-49ee-84c0-a88a8643b1b2 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:08.008404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3f378-6865-4f8a-b723-9a478d239c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:08.009030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3f378-6865-4f8a-b723-9a478d239c63 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:08.009311 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3f378-6865-4f8a-b723-9a478d239c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:08.009579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3f378-6865-4f8a-b723-9a478d239c63 None admin] PATH_INFO: `/v3/users/9025165ba2ca4fb69c65ca92d178890e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:08.010697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3f378-6865-4f8a-b723-9a478d239c63 None admin] RBAC: Authorizing `identity:get_user(user_id=9025165ba2ca4fb69c65ca92d178890e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:08.012165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3f378-6865-4f8a-b723-9a478d239c63 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:08.013764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 541/1096] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:07 2022] GET /identity/v3/users/9025165ba2ca4fb69c65ca92d178890e => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:08.020249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3f378-6865-4f8a-b723-9a478d239c63 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:08.041655 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4930d-3646-48f7-ab0c-f2c5646cdd0d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:08.075206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7b001-a7fb-41ec-9371-2dff7d13b725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:08.075804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7b001-a7fb-41ec-9371-2dff7d13b725 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:08.076111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7b001-a7fb-41ec-9371-2dff7d13b725 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:08.076386 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7b001-a7fb-41ec-9371-2dff7d13b725 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:08.077099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7b001-a7fb-41ec-9371-2dff7d13b725 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:08.077402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7b001-a7fb-41ec-9371-2dff7d13b725 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:08.077497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:08.078862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7b001-a7fb-41ec-9371-2dff7d13b725 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:08.095481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 556/1097] 10.222.0.33 () {56 vars in 1133 bytes} [Sun Sep 11 05:18:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:08.101127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7b001-a7fb-41ec-9371-2dff7d13b725 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:08.122582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3f378-6865-4f8a-b723-9a478d239c63 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:08.125755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:08.160903 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbff24-e936-40e3-b137-d387a5c08bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:08.161507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbff24-e936-40e3-b137-d387a5c08bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:08.161789 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbff24-e936-40e3-b137-d387a5c08bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:08.162060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbff24-e936-40e3-b137-d387a5c08bd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:08.198619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7b001-a7fb-41ec-9371-2dff7d13b725 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:08.228628 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbff24-e936-40e3-b137-d387a5c08bd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:08.230172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbff24-e936-40e3-b137-d387a5c08bd1 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:08.232129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d12dab-61bc-4bba-b24b-556ff6379a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:08.232732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d12dab-61bc-4bba-b24b-556ff6379a08 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:08.233010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d12dab-61bc-4bba-b24b-556ff6379a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:08.233280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d12dab-61bc-4bba-b24b-556ff6379a08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:08.285779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:08.301175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d12dab-61bc-4bba-b24b-556ff6379a08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:08.302637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d12dab-61bc-4bba-b24b-556ff6379a08 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:08.325423 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:08.327866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 542/1098] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:18:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 05:18:08.334718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbff24-e936-40e3-b137-d387a5c08bd1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:08.401866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 557/1099] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:18:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 05:18:08.408263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d12dab-61bc-4bba-b24b-556ff6379a08 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:08.436249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbff24-e936-40e3-b137-d387a5c08bd1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:08.471413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865703e6-10d5-4c16-9ef2-4a0f4c0eb754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:08.471954 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865703e6-10d5-4c16-9ef2-4a0f4c0eb754 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:08.472257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865703e6-10d5-4c16-9ef2-4a0f4c0eb754 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:08.472529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865703e6-10d5-4c16-9ef2-4a0f4c0eb754 None admin] PATH_INFO: `/v3/projects/0664a432c1024a8a82884e5b3ae76cb6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:08.473625 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865703e6-10d5-4c16-9ef2-4a0f4c0eb754 None admin] RBAC: Authorizing `identity:get_project(project_id=0664a432c1024a8a82884e5b3ae76cb6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:08.475040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865703e6-10d5-4c16-9ef2-4a0f4c0eb754 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:08.476677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 543/1100] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:18:08 2022] GET /identity/v3/projects/0664a432c1024a8a82884e5b3ae76cb6 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:08.482695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865703e6-10d5-4c16-9ef2-4a0f4c0eb754 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:08.504537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d12dab-61bc-4bba-b24b-556ff6379a08 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:08.510477 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:08.527426 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:08.539175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a999d4-4b57-4587-bcee-5c9363cde937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:08.539796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a999d4-4b57-4587-bcee-5c9363cde937 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:08.540099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a999d4-4b57-4587-bcee-5c9363cde937 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:08.540373 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a999d4-4b57-4587-bcee-5c9363cde937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:08.550732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a999d4-4b57-4587-bcee-5c9363cde937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:08.552670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a999d4-4b57-4587-bcee-5c9363cde937 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-57a999d4-4b57-4587-bcee-5c9363cde937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:18:08.570329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:08.571485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:18:08.572433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:18:08.572433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:18:08.572433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:18:08.572433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 558/1101] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:18:08.577305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a999d4-4b57-4587-bcee-5c9363cde937 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:08.584692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865703e6-10d5-4c16-9ef2-4a0f4c0eb754 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:08.618514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459b32e-4b8c-44ae-aef9-42df7e7ca50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:08.619110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459b32e-4b8c-44ae-aef9-42df7e7ca50c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:08.619384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459b32e-4b8c-44ae-aef9-42df7e7ca50c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:08.619652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459b32e-4b8c-44ae-aef9-42df7e7ca50c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:08.620397 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459b32e-4b8c-44ae-aef9-42df7e7ca50c None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=0664a432c1024a8a82884e5b3ae76cb6, user.id=9025165ba2ca4fb69c65ca92d178890e) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:08.620701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459b32e-4b8c-44ae-aef9-42df7e7ca50c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:08.622129 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459b32e-4b8c-44ae-aef9-42df7e7ca50c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:08.627979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 544/1102] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:18:08 2022] GET /identity/v3/role_assignments?user.id=9025165ba2ca4fb69c65ca92d178890e&scope.project.id=0664a432c1024a8a82884e5b3ae76cb6&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:08.680392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a999d4-4b57-4587-bcee-5c9363cde937 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:08.715733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff30845-43be-4157-a47b-10e05836d336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:08.716377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff30845-43be-4157-a47b-10e05836d336 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:08.716659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff30845-43be-4157-a47b-10e05836d336 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:08.716932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff30845-43be-4157-a47b-10e05836d336 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:08.717590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff30845-43be-4157-a47b-10e05836d336 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:08.717897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff30845-43be-4157-a47b-10e05836d336 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:08.719880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff30845-43be-4157-a47b-10e05836d336 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:08.732708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 559/1103] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:08.739024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459b32e-4b8c-44ae-aef9-42df7e7ca50c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:08.838931 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:08.849729 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459b32e-4b8c-44ae-aef9-42df7e7ca50c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:08.882922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358231f-7f38-4dc0-acd5-f1e25a331716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:08.883546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358231f-7f38-4dc0-acd5-f1e25a331716 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:08.883822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358231f-7f38-4dc0-acd5-f1e25a331716 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:08.884119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358231f-7f38-4dc0-acd5-f1e25a331716 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:08.891105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358231f-7f38-4dc0-acd5-f1e25a331716 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:08.892598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358231f-7f38-4dc0-acd5-f1e25a331716 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-1358231f-7f38-4dc0-acd5-f1e25a331716 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:18:08.903444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:08.904368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:08.905306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:18:08.905306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:18:08.905306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:18:08.905306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:18:08.905306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:18:08.905306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:18:08.905306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:18:08.905306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:18:08.905306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 545/1104] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:18:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:08.911079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff30845-43be-4157-a47b-10e05836d336 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:08.945127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:09.009812 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff30845-43be-4157-a47b-10e05836d336 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:09.042377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b52f1-d8c0-4cfc-85d7-1d47dfa17115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:09.042984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b52f1-d8c0-4cfc-85d7-1d47dfa17115 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:09.043266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b52f1-d8c0-4cfc-85d7-1d47dfa17115 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:09.043538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b52f1-d8c0-4cfc-85d7-1d47dfa17115 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:09.044262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b52f1-d8c0-4cfc-85d7-1d47dfa17115 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:09.044573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b52f1-d8c0-4cfc-85d7-1d47dfa17115 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:09.045988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b52f1-d8c0-4cfc-85d7-1d47dfa17115 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:09.057625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 560/1105] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:18:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:09.064482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358231f-7f38-4dc0-acd5-f1e25a331716 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:09.165801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358231f-7f38-4dc0-acd5-f1e25a331716 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:09.199156 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187026b-68ee-4307-9a10-c0c47c39460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:09.199750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187026b-68ee-4307-9a10-c0c47c39460d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:09.200051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187026b-68ee-4307-9a10-c0c47c39460d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:09.200368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187026b-68ee-4307-9a10-c0c47c39460d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:09.201085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187026b-68ee-4307-9a10-c0c47c39460d None admin] RBAC: Adding query filter params (role.id=a1192a1e2adb45d490e8d553aa87c6a6, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=3bd5a9a09f034c05815c631a44d1cdd9) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:09.201386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187026b-68ee-4307-9a10-c0c47c39460d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:09.202808 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187026b-68ee-4307-9a10-c0c47c39460d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:09.209152 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 546/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:18:09 2022] GET /identity/v3/role_assignments?user.id=3bd5a9a09f034c05815c631a44d1cdd9&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=a1192a1e2adb45d490e8d553aa87c6a6 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:10.812870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8b9cb-b361-4708-bf38-4e0960ad1fc7 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:10.813365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8b9cb-b361-4708-bf38-4e0960ad1fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:10.813481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8b9cb-b361-4708-bf38-4e0960ad1fc7 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:10.814735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 561/1107] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:10.820148 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77ab93-0c93-441c-8c57-ed847b4bded1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:10.820554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77ab93-0c93-441c-8c57-ed847b4bded1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:10.820694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77ab93-0c93-441c-8c57-ed847b4bded1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:10.838496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3c77ab93-0c93-441c-8c57-ed847b4bded1 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:10.839668 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3c77ab93-0c93-441c-8c57-ed847b4bded1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:10.849150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:10.906934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 547/1108] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:11.178736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8b9cb-b361-4708-bf38-4e0960ad1fc7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:11.273090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:11.280099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8b9cb-b361-4708-bf38-4e0960ad1fc7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:11.312649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b94c6-2edb-4023-867c-3ac1ccf8b6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:11.313258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b94c6-2edb-4023-867c-3ac1ccf8b6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:11.313540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b94c6-2edb-4023-867c-3ac1ccf8b6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:11.313806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b94c6-2edb-4023-867c-3ac1ccf8b6d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:11.314900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b94c6-2edb-4023-867c-3ac1ccf8b6d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:11.316340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b94c6-2edb-4023-867c-3ac1ccf8b6d6 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:11.317886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 562/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:18:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:11.323472 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77ab93-0c93-441c-8c57-ed847b4bded1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:11.420606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77ab93-0c93-441c-8c57-ed847b4bded1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:11.453736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c745b-2beb-4621-abfd-78525a87f311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:11.454267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c745b-2beb-4621-abfd-78525a87f311 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:11.454549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c745b-2beb-4621-abfd-78525a87f311 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:11.454822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c745b-2beb-4621-abfd-78525a87f311 None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:11.455469 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c745b-2beb-4621-abfd-78525a87f311 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:11.456921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c745b-2beb-4621-abfd-78525a87f311 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:11.473800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:11.479501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 548/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:18:11 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:18:13.647511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0f016-6186-4f1e-abb1-8fcd95664724 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:13.649489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0f016-6186-4f1e-abb1-8fcd95664724 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:13.649489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0f016-6186-4f1e-abb1-8fcd95664724 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:13.649489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 563/1111] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:13.654557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a7b8-87c7-4f76-997a-4693f2809200 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:13.654956 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a7b8-87c7-4f76-997a-4693f2809200 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:13.655104 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a7b8-87c7-4f76-997a-4693f2809200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:13.685673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0e18a7b8-87c7-4f76-997a-4693f2809200 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:13.686862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0e18a7b8-87c7-4f76-997a-4693f2809200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:13.753417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 549/1112] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:14.026814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0f016-6186-4f1e-abb1-8fcd95664724 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:14.123938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0f016-6186-4f1e-abb1-8fcd95664724 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:14.155890 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d022a34-a951-40cf-9858-1484d5490b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:14.156744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d022a34-a951-40cf-9858-1484d5490b0c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:14.157018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d022a34-a951-40cf-9858-1484d5490b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:14.157311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d022a34-a951-40cf-9858-1484d5490b0c None admin] PATH_INFO: `/v3/roles/6c6c7fa19ee342da915c90895e707620` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:14.158653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d022a34-a951-40cf-9858-1484d5490b0c None admin] RBAC: Authorizing `identity:get_role(role_id=6c6c7fa19ee342da915c90895e707620)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:14.160017 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d022a34-a951-40cf-9858-1484d5490b0c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:14.161388 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 564/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:14 2022] GET /identity/v3/roles/6c6c7fa19ee342da915c90895e707620 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:14.167080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18a7b8-87c7-4f76-997a-4693f2809200 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:14.262690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18a7b8-87c7-4f76-997a-4693f2809200 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:14.290908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:14.297830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173b94e-0163-4585-9dd4-37af82244878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:14.298456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173b94e-0163-4585-9dd4-37af82244878 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:14.298736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173b94e-0163-4585-9dd4-37af82244878 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:14.299008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173b94e-0163-4585-9dd4-37af82244878 None admin] PATH_INFO: `/v3/users/285d50bbacd743a6bd1772198e051581` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:14.311684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173b94e-0163-4585-9dd4-37af82244878 None admin] RBAC: Authorizing `identity:get_user(user_id=285d50bbacd743a6bd1772198e051581)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:14.313988 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173b94e-0163-4585-9dd4-37af82244878 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:14.315564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 550/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:14 2022] GET /identity/v3/users/285d50bbacd743a6bd1772198e051581 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:14.321329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d022a34-a951-40cf-9858-1484d5490b0c None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:14.354484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:14.422063 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d022a34-a951-40cf-9858-1484d5490b0c None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:14.454440 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4967c55-abf4-426f-a9c0-dbcdd9aed046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:14.455041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4967c55-abf4-426f-a9c0-dbcdd9aed046 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:14.455321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4967c55-abf4-426f-a9c0-dbcdd9aed046 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:14.455592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4967c55-abf4-426f-a9c0-dbcdd9aed046 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:14.520420 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4967c55-abf4-426f-a9c0-dbcdd9aed046 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:14.521862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4967c55-abf4-426f-a9c0-dbcdd9aed046 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:14.550811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:14.616753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 565/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:18:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:18:14.622681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173b94e-0163-4585-9dd4-37af82244878 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:14.720400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173b94e-0163-4585-9dd4-37af82244878 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:14.753589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6811c3-3483-4137-adfa-8a69433e2e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:14.754461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6811c3-3483-4137-adfa-8a69433e2e75 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:14.754744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6811c3-3483-4137-adfa-8a69433e2e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:14.755017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6811c3-3483-4137-adfa-8a69433e2e75 None admin] PATH_INFO: `/v3/projects/0664a432c1024a8a82884e5b3ae76cb6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:14.756151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6811c3-3483-4137-adfa-8a69433e2e75 None admin] RBAC: Authorizing `identity:get_project(project_id=0664a432c1024a8a82884e5b3ae76cb6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:14.757577 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6811c3-3483-4137-adfa-8a69433e2e75 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:14.759174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 551/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:18:14 2022] GET /identity/v3/projects/0664a432c1024a8a82884e5b3ae76cb6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:14.765181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4967c55-abf4-426f-a9c0-dbcdd9aed046 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:14.860424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4967c55-abf4-426f-a9c0-dbcdd9aed046 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:14.888506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:14.894253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52d96b-9f62-4644-b24a-742c196eb4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:14.894842 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52d96b-9f62-4644-b24a-742c196eb4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:14.895116 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52d96b-9f62-4644-b24a-742c196eb4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:14.895385 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52d96b-9f62-4644-b24a-742c196eb4d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:14.896123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52d96b-9f62-4644-b24a-742c196eb4d4 None admin] RBAC: Adding query filter params (role.id=6c6c7fa19ee342da915c90895e707620, scope.project.id=0664a432c1024a8a82884e5b3ae76cb6, user.id=285d50bbacd743a6bd1772198e051581) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:14.896428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52d96b-9f62-4644-b24a-742c196eb4d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:14.897833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52d96b-9f62-4644-b24a-742c196eb4d4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:14.903014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 566/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:18:14 2022] GET /identity/v3/role_assignments?user.id=285d50bbacd743a6bd1772198e051581&scope.project.id=0664a432c1024a8a82884e5b3ae76cb6&role.id=6c6c7fa19ee342da915c90895e707620 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:17.010403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392a393-4ee8-4c9c-bcc9-92937c07e20c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:17.010922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392a393-4ee8-4c9c-bcc9-92937c07e20c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:17.010922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392a393-4ee8-4c9c-bcc9-92937c07e20c None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:17.012309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 552/1118] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:17.017603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec90ef-d662-4ebb-8cc7-2ce12cef3dd3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:17.018040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec90ef-d662-4ebb-8cc7-2ce12cef3dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:17.018141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec90ef-d662-4ebb-8cc7-2ce12cef3dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:17.036441 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-59ec90ef-d662-4ebb-8cc7-2ce12cef3dd3 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:17.037616 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-59ec90ef-d662-4ebb-8cc7-2ce12cef3dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:17.101864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 567/1119] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:17.370913 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392a393-4ee8-4c9c-bcc9-92937c07e20c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:17.428413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:17.472447 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392a393-4ee8-4c9c-bcc9-92937c07e20c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:17.506373 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2196b-4304-4d30-af3a-9e400d5d409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:17.506990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2196b-4304-4d30-af3a-9e400d5d409b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:17.507269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2196b-4304-4d30-af3a-9e400d5d409b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:17.507540 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2196b-4304-4d30-af3a-9e400d5d409b None admin] PATH_INFO: `/v3/roles/6c6c7fa19ee342da915c90895e707620` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:17.508926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2196b-4304-4d30-af3a-9e400d5d409b None admin] RBAC: Authorizing `identity:get_role(role_id=6c6c7fa19ee342da915c90895e707620)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:17.510291 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2196b-4304-4d30-af3a-9e400d5d409b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:17.511593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 553/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:17 2022] GET /identity/v3/roles/6c6c7fa19ee342da915c90895e707620 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:17.517455 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec90ef-d662-4ebb-8cc7-2ce12cef3dd3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:17.611923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec90ef-d662-4ebb-8cc7-2ce12cef3dd3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:17.635871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:17.644569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e82f81-31ad-403c-bd2d-56888a8fd2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:17.645180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e82f81-31ad-403c-bd2d-56888a8fd2fc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:17.645455 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e82f81-31ad-403c-bd2d-56888a8fd2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:17.645720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e82f81-31ad-403c-bd2d-56888a8fd2fc None admin] PATH_INFO: `/v3/users/285d50bbacd743a6bd1772198e051581` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:17.646802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e82f81-31ad-403c-bd2d-56888a8fd2fc None admin] RBAC: Authorizing `identity:get_user(user_id=285d50bbacd743a6bd1772198e051581)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:17.648251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e82f81-31ad-403c-bd2d-56888a8fd2fc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:17.649781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 568/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:17 2022] GET /identity/v3/users/285d50bbacd743a6bd1772198e051581 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:17.655451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2196b-4304-4d30-af3a-9e400d5d409b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:17.753388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2196b-4304-4d30-af3a-9e400d5d409b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:17.776833 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:17.788136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06c706-906a-4c8e-a76b-fdcf3566f0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:17.788757 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06c706-906a-4c8e-a76b-fdcf3566f0cb None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:17.789032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06c706-906a-4c8e-a76b-fdcf3566f0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:17.789302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06c706-906a-4c8e-a76b-fdcf3566f0cb None admin] PATH_INFO: `/v3/projects/0664a432c1024a8a82884e5b3ae76cb6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:17.790421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06c706-906a-4c8e-a76b-fdcf3566f0cb None admin] RBAC: Authorizing `identity:get_project(project_id=0664a432c1024a8a82884e5b3ae76cb6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:17.791842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06c706-906a-4c8e-a76b-fdcf3566f0cb None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:17.793547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 554/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:18:17 2022] GET /identity/v3/projects/0664a432c1024a8a82884e5b3ae76cb6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:17.800251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e82f81-31ad-403c-bd2d-56888a8fd2fc None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:17.895101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e82f81-31ad-403c-bd2d-56888a8fd2fc None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:17.927932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b1ea7-1c6b-4643-9980-de85a371efde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:17.928611 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b1ea7-1c6b-4643-9980-de85a371efde None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:17.928891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b1ea7-1c6b-4643-9980-de85a371efde None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:17.929164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b1ea7-1c6b-4643-9980-de85a371efde None admin] PATH_INFO: `/v3/projects/0664a432c1024a8a82884e5b3ae76cb6/users/285d50bbacd743a6bd1772198e051581/roles/6c6c7fa19ee342da915c90895e707620` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:17.931269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b1ea7-1c6b-4643-9980-de85a371efde None admin] RBAC: Authorizing `identity:create_grant(project_id=0664a432c1024a8a82884e5b3ae76cb6, user_id=285d50bbacd743a6bd1772198e051581, role_id=6c6c7fa19ee342da915c90895e707620)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:17.932725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b1ea7-1c6b-4643-9980-de85a371efde None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:17.945161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 569/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:18:17 2022] PUT /identity/v3/projects/0664a432c1024a8a82884e5b3ae76cb6/users/285d50bbacd743a6bd1772198e051581/roles/6c6c7fa19ee342da915c90895e707620 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:18:20.088782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4b2fe-d316-4d81-8848-b4e9d5d1c7c6 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:20.089313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4b2fe-d316-4d81-8848-b4e9d5d1c7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:20.089313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4b2fe-d316-4d81-8848-b4e9d5d1c7c6 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:20.090658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 555/1124] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:20.096173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b917b-b8e4-4240-8481-a3573f83026a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:20.096560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b917b-b8e4-4240-8481-a3573f83026a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:20.096718 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b917b-b8e4-4240-8481-a3573f83026a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:20.114980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1e5b917b-b8e4-4240-8481-a3573f83026a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:20.116179 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1e5b917b-b8e4-4240-8481-a3573f83026a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:20.164086 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:20.182603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 570/1125] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:20.453883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4b2fe-d316-4d81-8848-b4e9d5d1c7c6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:20.553712 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4b2fe-d316-4d81-8848-b4e9d5d1c7c6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:20.586654 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd9e11-33cc-4fd9-a47d-59a993441941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:20.587264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd9e11-33cc-4fd9-a47d-59a993441941 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:20.587540 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd9e11-33cc-4fd9-a47d-59a993441941 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:20.587811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd9e11-33cc-4fd9-a47d-59a993441941 None admin] PATH_INFO: `/v3/roles/6c6c7fa19ee342da915c90895e707620` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:20.589197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd9e11-33cc-4fd9-a47d-59a993441941 None admin] RBAC: Authorizing `identity:get_role(role_id=6c6c7fa19ee342da915c90895e707620)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:20.590564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd9e11-33cc-4fd9-a47d-59a993441941 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:20.591855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 556/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:20 2022] GET /identity/v3/roles/6c6c7fa19ee342da915c90895e707620 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:20.598195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b917b-b8e4-4240-8481-a3573f83026a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:20.692731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b917b-b8e4-4240-8481-a3573f83026a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:20.725064 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ff435-7c78-4042-9ddc-faf3367ea2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:20.725624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ff435-7c78-4042-9ddc-faf3367ea2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:20.725900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ff435-7c78-4042-9ddc-faf3367ea2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:20.726166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ff435-7c78-4042-9ddc-faf3367ea2b1 None admin] PATH_INFO: `/v3/users/285d50bbacd743a6bd1772198e051581` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:20.727266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ff435-7c78-4042-9ddc-faf3367ea2b1 None admin] RBAC: Authorizing `identity:get_user(user_id=285d50bbacd743a6bd1772198e051581)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:20.728715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ff435-7c78-4042-9ddc-faf3367ea2b1 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:20.730274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 571/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:20 2022] GET /identity/v3/users/285d50bbacd743a6bd1772198e051581 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:20.736822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd9e11-33cc-4fd9-a47d-59a993441941 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:20.804719 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:20.845193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd9e11-33cc-4fd9-a47d-59a993441941 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:20.877598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e269bb1-9542-4670-aafb-a53050aaea35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:20.878198 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e269bb1-9542-4670-aafb-a53050aaea35 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:20.878471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e269bb1-9542-4670-aafb-a53050aaea35 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:20.878738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e269bb1-9542-4670-aafb-a53050aaea35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:20.945384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e269bb1-9542-4670-aafb-a53050aaea35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:20.946825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e269bb1-9542-4670-aafb-a53050aaea35 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:21.003058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:21.044106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 557/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:18:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:18:21.050062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ff435-7c78-4042-9ddc-faf3367ea2b1 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:21.101831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:21.147307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ff435-7c78-4042-9ddc-faf3367ea2b1 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:21.180383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0cbd3b-f377-4415-ac01-d703511b1f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:21.180999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0cbd3b-f377-4415-ac01-d703511b1f29 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:21.181278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0cbd3b-f377-4415-ac01-d703511b1f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:21.181547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0cbd3b-f377-4415-ac01-d703511b1f29 None admin] PATH_INFO: `/v3/projects/0664a432c1024a8a82884e5b3ae76cb6` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:21.182640 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0cbd3b-f377-4415-ac01-d703511b1f29 None admin] RBAC: Authorizing `identity:get_project(project_id=0664a432c1024a8a82884e5b3ae76cb6)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:21.184185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0cbd3b-f377-4415-ac01-d703511b1f29 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:21.185800 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 572/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:18:21 2022] GET /identity/v3/projects/0664a432c1024a8a82884e5b3ae76cb6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:21.193059 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e269bb1-9542-4670-aafb-a53050aaea35 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:21.289863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e269bb1-9542-4670-aafb-a53050aaea35 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:21.323629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632de36-7ef6-41b2-b2d0-e72784e47975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:21.324244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632de36-7ef6-41b2-b2d0-e72784e47975 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:21.324529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632de36-7ef6-41b2-b2d0-e72784e47975 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:21.324799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632de36-7ef6-41b2-b2d0-e72784e47975 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:21.325510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632de36-7ef6-41b2-b2d0-e72784e47975 None admin] RBAC: Adding query filter params (role.id=6c6c7fa19ee342da915c90895e707620, scope.project.id=0664a432c1024a8a82884e5b3ae76cb6, user.id=285d50bbacd743a6bd1772198e051581) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:21.325814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632de36-7ef6-41b2-b2d0-e72784e47975 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:21.327230 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632de36-7ef6-41b2-b2d0-e72784e47975 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:21.334276 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 558/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:18:21 2022] GET /identity/v3/role_assignments?user.id=285d50bbacd743a6bd1772198e051581&scope.project.id=0664a432c1024a8a82884e5b3ae76cb6&role.id=6c6c7fa19ee342da915c90895e707620 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:23.501723 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdaac8d-2c26-440b-83f2-16c28153337e None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:23.502258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdaac8d-2c26-440b-83f2-16c28153337e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:23.502258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdaac8d-2c26-440b-83f2-16c28153337e None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:23.503605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 573/1131] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:23.509162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b202d5-1603-4aee-995c-dc773d241bea None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:23.509560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b202d5-1603-4aee-995c-dc773d241bea None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:23.509704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b202d5-1603-4aee-995c-dc773d241bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:23.527863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c1b202d5-1603-4aee-995c-dc773d241bea None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:23.529063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c1b202d5-1603-4aee-995c-dc773d241bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:23.545750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:23.594547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 559/1132] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:23.864093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdaac8d-2c26-440b-83f2-16c28153337e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:23.961515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdaac8d-2c26-440b-83f2-16c28153337e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:23.980893 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:23.994611 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e0c4d-769d-4c81-8c39-ae1feade96bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:23.995223 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e0c4d-769d-4c81-8c39-ae1feade96bc None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:23.995498 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e0c4d-769d-4c81-8c39-ae1feade96bc None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:23.995762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e0c4d-769d-4c81-8c39-ae1feade96bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:23.996872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e0c4d-769d-4c81-8c39-ae1feade96bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:23.998281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e0c4d-769d-4c81-8c39-ae1feade96bc None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:24.000170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 574/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:18:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:24.006059 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b202d5-1603-4aee-995c-dc773d241bea None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:24.102628 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b202d5-1603-4aee-995c-dc773d241bea None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:24.135343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a3f8f-96be-41d2-a134-34e65221b738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:24.135933 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a3f8f-96be-41d2-a134-34e65221b738 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:24.136239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a3f8f-96be-41d2-a134-34e65221b738 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:24.136509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a3f8f-96be-41d2-a134-34e65221b738 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:24.137141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a3f8f-96be-41d2-a134-34e65221b738 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:24.138566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a3f8f-96be-41d2-a134-34e65221b738 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:24.164344 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 560/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:18:24 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:18:26.347517 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81258b20-547e-46d7-8053-ed4156ee5d76 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:26.348110 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81258b20-547e-46d7-8053-ed4156ee5d76 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:26.348110 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81258b20-547e-46d7-8053-ed4156ee5d76 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:26.349396 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 575/1135] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:26.354711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da4a9e-5132-469e-ac14-e3f4bb414db1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:26.355082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da4a9e-5132-469e-ac14-e3f4bb414db1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:26.355268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da4a9e-5132-469e-ac14-e3f4bb414db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:26.369302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:26.375780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d1da4a9e-5132-469e-ac14-e3f4bb414db1 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:26.377490 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d1da4a9e-5132-469e-ac14-e3f4bb414db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:26.442676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 561/1136] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:26.711950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81258b20-547e-46d7-8053-ed4156ee5d76 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:26.810877 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81258b20-547e-46d7-8053-ed4156ee5d76 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:26.844382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0015002-40ff-4459-a412-ff093970381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:26.845006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0015002-40ff-4459-a412-ff093970381d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:26.845284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0015002-40ff-4459-a412-ff093970381d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:26.845555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0015002-40ff-4459-a412-ff093970381d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:26.846656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0015002-40ff-4459-a412-ff093970381d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:26.848097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0015002-40ff-4459-a412-ff093970381d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:26.849653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 576/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:18:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:26.855291 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da4a9e-5132-469e-ac14-e3f4bb414db1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:26.950462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da4a9e-5132-469e-ac14-e3f4bb414db1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:26.984073 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e3936-3048-4079-9f73-32d569ffb13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:26.984614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e3936-3048-4079-9f73-32d569ffb13a None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:26.984889 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e3936-3048-4079-9f73-32d569ffb13a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:26.985155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e3936-3048-4079-9f73-32d569ffb13a None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:26.985785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e3936-3048-4079-9f73-32d569ffb13a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:26.987202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e3936-3048-4079-9f73-32d569ffb13a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:26.995981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:27.007890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 562/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:18:26 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:18:29.185725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7b7a4-cd87-456e-aa37-87482d539587 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:29.186288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7b7a4-cd87-456e-aa37-87482d539587 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:29.186288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7b7a4-cd87-456e-aa37-87482d539587 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:29.187685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 577/1139] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:29.193011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da936d2b-ce50-4d5f-874c-ab284a133422 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:29.193450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da936d2b-ce50-4d5f-874c-ab284a133422 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:29.193572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da936d2b-ce50-4d5f-874c-ab284a133422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:29.212125 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-da936d2b-ce50-4d5f-874c-ab284a133422 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:29.213294 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-da936d2b-ce50-4d5f-874c-ab284a133422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:29.278108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 563/1140] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:29.549466 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7b7a4-cd87-456e-aa37-87482d539587 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:29.608714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:29.651919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7b7a4-cd87-456e-aa37-87482d539587 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:29.684073 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8c9ff-d766-4a74-9250-0c54191a8611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:29.684685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8c9ff-d766-4a74-9250-0c54191a8611 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:29.684963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8c9ff-d766-4a74-9250-0c54191a8611 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:29.685231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8c9ff-d766-4a74-9250-0c54191a8611 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:29.695346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8c9ff-d766-4a74-9250-0c54191a8611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5aa8c9ff-d766-4a74-9250-0c54191a8611 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:18:29.700096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:29.701301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:29.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:29.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:29.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:29.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:29.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:18:29.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:18:29.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:18:29.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:18:29.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:18:29.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:18:29.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:29.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:18:29.702392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 578/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:18:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:29.707684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da936d2b-ce50-4d5f-874c-ab284a133422 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:29.805449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da936d2b-ce50-4d5f-874c-ab284a133422 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:29.828324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:29.840384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b60bf-6e24-4d43-8a3b-ad6e4940bbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:29.840987 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b60bf-6e24-4d43-8a3b-ad6e4940bbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:29.841267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b60bf-6e24-4d43-8a3b-ad6e4940bbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:29.841539 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b60bf-6e24-4d43-8a3b-ad6e4940bbf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:29.842196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b60bf-6e24-4d43-8a3b-ad6e4940bbf9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:29.842499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b60bf-6e24-4d43-8a3b-ad6e4940bbf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:29.844020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b60bf-6e24-4d43-8a3b-ad6e4940bbf9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:29.851852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 564/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:18:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:29.857798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8c9ff-d766-4a74-9250-0c54191a8611 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:29.954156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8c9ff-d766-4a74-9250-0c54191a8611 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:29.969539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:29.987400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795701d9-ae40-440a-b321-9d7525c9720d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:29.988028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795701d9-ae40-440a-b321-9d7525c9720d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:29.988304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795701d9-ae40-440a-b321-9d7525c9720d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:29.988573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795701d9-ae40-440a-b321-9d7525c9720d None admin] PATH_INFO: `/v3/users/d8798cf756f24ba39d4a2eb7e2720a48` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:30.000980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795701d9-ae40-440a-b321-9d7525c9720d None admin] RBAC: Authorizing `identity:get_user(user_id=d8798cf756f24ba39d4a2eb7e2720a48)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:30.002395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795701d9-ae40-440a-b321-9d7525c9720d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:30.003945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 579/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:29 2022] GET /identity/v3/users/d8798cf756f24ba39d4a2eb7e2720a48 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:30.009927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b60bf-6e24-4d43-8a3b-ad6e4940bbf9 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:30.108153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b60bf-6e24-4d43-8a3b-ad6e4940bbf9 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:30.142293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51521-eff0-4fba-8d23-3829b1ec645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:30.142885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51521-eff0-4fba-8d23-3829b1ec645c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:30.143162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51521-eff0-4fba-8d23-3829b1ec645c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:30.143427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c51521-eff0-4fba-8d23-3829b1ec645c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:30.194149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:30.210295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c51521-eff0-4fba-8d23-3829b1ec645c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:30.211731 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c51521-eff0-4fba-8d23-3829b1ec645c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:30.307730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 565/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:18:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:18:30.314158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795701d9-ae40-440a-b321-9d7525c9720d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:30.409588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795701d9-ae40-440a-b321-9d7525c9720d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:30.442192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e855ac-d9b6-4edc-8c20-86cbe599b0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:30.442804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e855ac-d9b6-4edc-8c20-86cbe599b0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:30.443081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e855ac-d9b6-4edc-8c20-86cbe599b0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:30.443349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e855ac-d9b6-4edc-8c20-86cbe599b0d7 None admin] PATH_INFO: `/v3/projects/2e035e5df9d341a7837b99a99ead31b3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:30.444465 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e855ac-d9b6-4edc-8c20-86cbe599b0d7 None admin] RBAC: Authorizing `identity:get_project(project_id=2e035e5df9d341a7837b99a99ead31b3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:30.445886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e855ac-d9b6-4edc-8c20-86cbe599b0d7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:30.447473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 580/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:18:30 2022] GET /identity/v3/projects/2e035e5df9d341a7837b99a99ead31b3 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:30.454051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51521-eff0-4fba-8d23-3829b1ec645c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:30.535337 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:30.551468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c51521-eff0-4fba-8d23-3829b1ec645c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:30.584575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8ff3b-66ac-4c02-b241-7c5c8ceed4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:30.585159 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8ff3b-66ac-4c02-b241-7c5c8ceed4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:30.585433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8ff3b-66ac-4c02-b241-7c5c8ceed4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:30.585700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8ff3b-66ac-4c02-b241-7c5c8ceed4d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:30.586406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8ff3b-66ac-4c02-b241-7c5c8ceed4d6 None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=2e035e5df9d341a7837b99a99ead31b3, user.id=d8798cf756f24ba39d4a2eb7e2720a48) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:30.586708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8ff3b-66ac-4c02-b241-7c5c8ceed4d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:30.588149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8ff3b-66ac-4c02-b241-7c5c8ceed4d6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:30.593556 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 566/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:18:30 2022] GET /identity/v3/role_assignments?user.id=d8798cf756f24ba39d4a2eb7e2720a48&scope.project.id=2e035e5df9d341a7837b99a99ead31b3&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:32.700540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67504e3a-90a5-4ffb-aeb4-a6dc331ee7b8 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:32.701099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67504e3a-90a5-4ffb-aeb4-a6dc331ee7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:32.701099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67504e3a-90a5-4ffb-aeb4-a6dc331ee7b8 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:32.702971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 581/1147] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:32.708022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41266f8f-9700-49f2-9576-3ba393913d2a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:32.708445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41266f8f-9700-49f2-9576-3ba393913d2a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:32.708570 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41266f8f-9700-49f2-9576-3ba393913d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:32.726119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-41266f8f-9700-49f2-9576-3ba393913d2a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:32.727288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-41266f8f-9700-49f2-9576-3ba393913d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:32.794579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 567/1148] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:33.060428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67504e3a-90a5-4ffb-aeb4-a6dc331ee7b8 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:33.103268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:33.158730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67504e3a-90a5-4ffb-aeb4-a6dc331ee7b8 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:33.190909 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266332e-21cd-4880-b8cb-068b27a3e28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:33.191516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266332e-21cd-4880-b8cb-068b27a3e28e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:33.191791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266332e-21cd-4880-b8cb-068b27a3e28e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:33.192091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266332e-21cd-4880-b8cb-068b27a3e28e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:33.201611 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266332e-21cd-4880-b8cb-068b27a3e28e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-7266332e-21cd-4880-b8cb-068b27a3e28e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:18:33.206421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:33.207580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:33.208874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:33.208874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:33.208874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:33.208874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:33.208874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:18:33.208874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:18:33.208874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:18:33.208874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:18:33.208874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:18:33.208874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:18:33.208874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:33.208874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:18:33.208874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 582/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:18:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:33.213423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41266f8f-9700-49f2-9576-3ba393913d2a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:33.289495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:33.311323 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41266f8f-9700-49f2-9576-3ba393913d2a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:33.344877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32fadd-7f41-4224-9de6-0ce24cd1f620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:33.345470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32fadd-7f41-4224-9de6-0ce24cd1f620 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:33.345752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32fadd-7f41-4224-9de6-0ce24cd1f620 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:33.346027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32fadd-7f41-4224-9de6-0ce24cd1f620 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:33.346690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32fadd-7f41-4224-9de6-0ce24cd1f620 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:33.346995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32fadd-7f41-4224-9de6-0ce24cd1f620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:33.348390 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32fadd-7f41-4224-9de6-0ce24cd1f620 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:33.356203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 568/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:18:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:33.362781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266332e-21cd-4880-b8cb-068b27a3e28e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:33.458828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266332e-21cd-4880-b8cb-068b27a3e28e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:33.460329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:33.492856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2786a5-83c3-441c-8533-677cb3a37fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:33.493468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2786a5-83c3-441c-8533-677cb3a37fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:33.493748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2786a5-83c3-441c-8533-677cb3a37fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:33.494017 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2786a5-83c3-441c-8533-677cb3a37fd4 None admin] PATH_INFO: `/v3/users/d8798cf756f24ba39d4a2eb7e2720a48` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:33.495116 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2786a5-83c3-441c-8533-677cb3a37fd4 None admin] RBAC: Authorizing `identity:get_user(user_id=d8798cf756f24ba39d4a2eb7e2720a48)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:33.496568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2786a5-83c3-441c-8533-677cb3a37fd4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:33.498292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 583/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:33 2022] GET /identity/v3/users/d8798cf756f24ba39d4a2eb7e2720a48 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:33.504518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32fadd-7f41-4224-9de6-0ce24cd1f620 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:33.603021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32fadd-7f41-4224-9de6-0ce24cd1f620 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:33.635620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45566c-44e8-4792-8bdf-de1319e1bc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:33.636264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45566c-44e8-4792-8bdf-de1319e1bc56 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:33.636546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45566c-44e8-4792-8bdf-de1319e1bc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:33.636820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45566c-44e8-4792-8bdf-de1319e1bc56 None admin] PATH_INFO: `/v3/projects/2e035e5df9d341a7837b99a99ead31b3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:33.637935 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45566c-44e8-4792-8bdf-de1319e1bc56 None admin] RBAC: Authorizing `identity:get_project(project_id=2e035e5df9d341a7837b99a99ead31b3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:33.639360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45566c-44e8-4792-8bdf-de1319e1bc56 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:33.641714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 569/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:18:33 2022] GET /identity/v3/projects/2e035e5df9d341a7837b99a99ead31b3 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:33.647530 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2786a5-83c3-441c-8533-677cb3a37fd4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:33.743594 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2786a5-83c3-441c-8533-677cb3a37fd4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:33.777421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd9978-4ea9-463a-a37b-010b5f7adcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:33.778073 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd9978-4ea9-463a-a37b-010b5f7adcd3 None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:33.778356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd9978-4ea9-463a-a37b-010b5f7adcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:33.778630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd9978-4ea9-463a-a37b-010b5f7adcd3 None admin] PATH_INFO: `/v3/projects/2e035e5df9d341a7837b99a99ead31b3/users/d8798cf756f24ba39d4a2eb7e2720a48/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:33.780595 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd9978-4ea9-463a-a37b-010b5f7adcd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e035e5df9d341a7837b99a99ead31b3, user_id=d8798cf756f24ba39d4a2eb7e2720a48, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:33.782133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd9978-4ea9-463a-a37b-010b5f7adcd3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:33.800034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 584/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:18:33 2022] PUT /identity/v3/projects/2e035e5df9d341a7837b99a99ead31b3/users/d8798cf756f24ba39d4a2eb7e2720a48/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:18:35.895292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db076965-e642-4d92-a4b9-b54be5298f97 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:35.895828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db076965-e642-4d92-a4b9-b54be5298f97 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:35.895828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db076965-e642-4d92-a4b9-b54be5298f97 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:35.897216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 570/1154] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:35.902276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a369742-da7a-484e-a571-2fe4c3b303e7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:35.902698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a369742-da7a-484e-a571-2fe4c3b303e7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:35.902814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a369742-da7a-484e-a571-2fe4c3b303e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:35.922408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1a369742-da7a-484e-a571-2fe4c3b303e7 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:35.923574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1a369742-da7a-484e-a571-2fe4c3b303e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:35.946419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:35.988302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 585/1155] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:36.252379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db076965-e642-4d92-a4b9-b54be5298f97 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:36.263963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:36.355119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db076965-e642-4d92-a4b9-b54be5298f97 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:36.388007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b548b-41e9-47f7-8033-7ed1f10031d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:36.388644 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b548b-41e9-47f7-8033-7ed1f10031d1 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:36.388931 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b548b-41e9-47f7-8033-7ed1f10031d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:36.389206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b548b-41e9-47f7-8033-7ed1f10031d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:36.400260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b548b-41e9-47f7-8033-7ed1f10031d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ca2b548b-41e9-47f7-8033-7ed1f10031d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:18:36.405014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:36.406471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:36.407527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:36.407527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:36.407527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:36.407527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:36.407527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:18:36.407527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:18:36.407527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:18:36.407527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:18:36.407527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:18:36.407527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:18:36.407527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:36.407527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:18:36.407527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 571/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:18:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:36.412463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a369742-da7a-484e-a571-2fe4c3b303e7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:36.507509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a369742-da7a-484e-a571-2fe4c3b303e7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:36.539743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ab541-5a53-4bf1-acfe-a16decc7c1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:36.540357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ab541-5a53-4bf1-acfe-a16decc7c1ca None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:36.540631 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ab541-5a53-4bf1-acfe-a16decc7c1ca None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:36.540896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ab541-5a53-4bf1-acfe-a16decc7c1ca None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:36.541543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ab541-5a53-4bf1-acfe-a16decc7c1ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:36.541838 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ab541-5a53-4bf1-acfe-a16decc7c1ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:36.543181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ab541-5a53-4bf1-acfe-a16decc7c1ca None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:36.551593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 586/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:18:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:36.557385 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b548b-41e9-47f7-8033-7ed1f10031d1 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:36.623604 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:36.657954 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b548b-41e9-47f7-8033-7ed1f10031d1 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:36.690586 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bca74e-2664-473c-bed1-bd9893475071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:36.691200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bca74e-2664-473c-bed1-bd9893475071 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:36.691480 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bca74e-2664-473c-bed1-bd9893475071 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:36.691753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bca74e-2664-473c-bed1-bd9893475071 None admin] PATH_INFO: `/v3/users/d8798cf756f24ba39d4a2eb7e2720a48` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:36.692887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bca74e-2664-473c-bed1-bd9893475071 None admin] RBAC: Authorizing `identity:get_user(user_id=d8798cf756f24ba39d4a2eb7e2720a48)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:36.694397 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bca74e-2664-473c-bed1-bd9893475071 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:36.695952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 572/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:36 2022] GET /identity/v3/users/d8798cf756f24ba39d4a2eb7e2720a48 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:36.701599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ab541-5a53-4bf1-acfe-a16decc7c1ca None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:36.726822 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:36.800081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ab541-5a53-4bf1-acfe-a16decc7c1ca None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:36.832846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348b4d6-bbfb-45fd-8bad-6877adf24752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:36.833452 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348b4d6-bbfb-45fd-8bad-6877adf24752 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:36.833731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348b4d6-bbfb-45fd-8bad-6877adf24752 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:36.834003 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348b4d6-bbfb-45fd-8bad-6877adf24752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:36.898383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348b4d6-bbfb-45fd-8bad-6877adf24752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:36.899829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348b4d6-bbfb-45fd-8bad-6877adf24752 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:36.922273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:36.995656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 587/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:18:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:18:37.001679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bca74e-2664-473c-bed1-bd9893475071 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:37.099531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bca74e-2664-473c-bed1-bd9893475071 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:37.132114 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef420a10-992a-4d23-a914-d0b419cbbafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:37.132729 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef420a10-992a-4d23-a914-d0b419cbbafe None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:37.133007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef420a10-992a-4d23-a914-d0b419cbbafe None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:37.133277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef420a10-992a-4d23-a914-d0b419cbbafe None admin] PATH_INFO: `/v3/projects/2e035e5df9d341a7837b99a99ead31b3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:37.134371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef420a10-992a-4d23-a914-d0b419cbbafe None admin] RBAC: Authorizing `identity:get_project(project_id=2e035e5df9d341a7837b99a99ead31b3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:37.135813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef420a10-992a-4d23-a914-d0b419cbbafe None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:37.137440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 573/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:18:37 2022] GET /identity/v3/projects/2e035e5df9d341a7837b99a99ead31b3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:37.143804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348b4d6-bbfb-45fd-8bad-6877adf24752 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:37.238370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348b4d6-bbfb-45fd-8bad-6877adf24752 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:37.259768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:37.271608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03154155-c695-4cb0-a54b-748a2df152b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:37.272228 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03154155-c695-4cb0-a54b-748a2df152b9 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:37.272547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03154155-c695-4cb0-a54b-748a2df152b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:37.272816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03154155-c695-4cb0-a54b-748a2df152b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:37.273523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03154155-c695-4cb0-a54b-748a2df152b9 None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=2e035e5df9d341a7837b99a99ead31b3, user.id=d8798cf756f24ba39d4a2eb7e2720a48) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:37.273825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03154155-c695-4cb0-a54b-748a2df152b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:37.275222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03154155-c695-4cb0-a54b-748a2df152b9 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:37.281351 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 588/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:18:37 2022] GET /identity/v3/role_assignments?user.id=d8798cf756f24ba39d4a2eb7e2720a48&scope.project.id=2e035e5df9d341a7837b99a99ead31b3&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:39.636435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb53a8-747b-438e-b6b4-41261b350469 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:39.636968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb53a8-747b-438e-b6b4-41261b350469 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:39.636968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb53a8-747b-438e-b6b4-41261b350469 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:39.638310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 574/1162] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:39.644982 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6df9b-4b2c-4866-aa69-c61751a44e4c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:39.645417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6df9b-4b2c-4866-aa69-c61751a44e4c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:39.645534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6df9b-4b2c-4866-aa69-c61751a44e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:39.664056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8be6df9b-4b2c-4866-aa69-c61751a44e4c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:39.665250 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8be6df9b-4b2c-4866-aa69-c61751a44e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:39.730206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 589/1163] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:40.010068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb53a8-747b-438e-b6b4-41261b350469 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:40.013843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:40.111042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb53a8-747b-438e-b6b4-41261b350469 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:40.144515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d608b38-5925-4492-b5a9-84880bf9efc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:40.145112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d608b38-5925-4492-b5a9-84880bf9efc8 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:40.145392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d608b38-5925-4492-b5a9-84880bf9efc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:40.145664 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d608b38-5925-4492-b5a9-84880bf9efc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:40.215709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d608b38-5925-4492-b5a9-84880bf9efc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:40.217182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d608b38-5925-4492-b5a9-84880bf9efc8 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:40.219963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:40.314384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 575/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:18:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:18:40.320244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6df9b-4b2c-4866-aa69-c61751a44e4c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:40.413478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6df9b-4b2c-4866-aa69-c61751a44e4c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:40.429245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:40.446882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29301bd-28ab-49ef-8b6d-d088249635e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:40.447497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29301bd-28ab-49ef-8b6d-d088249635e0 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:40.447773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29301bd-28ab-49ef-8b6d-d088249635e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:40.448066 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29301bd-28ab-49ef-8b6d-d088249635e0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:40.455222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29301bd-28ab-49ef-8b6d-d088249635e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:40.456679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29301bd-28ab-49ef-8b6d-d088249635e0 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d29301bd-28ab-49ef-8b6d-d088249635e0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:40.467509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:40.469605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:18:40.470501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 590/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 05:18:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:40.475161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d608b38-5925-4492-b5a9-84880bf9efc8 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:40.571083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d608b38-5925-4492-b5a9-84880bf9efc8 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:40.573383 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:40.605690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9288fac-9759-4d9a-8c84-3a96b7fcda6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:40.606283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9288fac-9759-4d9a-8c84-3a96b7fcda6f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:40.606563 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9288fac-9759-4d9a-8c84-3a96b7fcda6f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:40.606839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9288fac-9759-4d9a-8c84-3a96b7fcda6f None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:40.607467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9288fac-9759-4d9a-8c84-3a96b7fcda6f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:40.607767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9288fac-9759-4d9a-8c84-3a96b7fcda6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:40.609264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9288fac-9759-4d9a-8c84-3a96b7fcda6f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:40.616171 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 576/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Sun Sep 11 05:18:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:42.758897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7edac-1018-476a-ad15-f5fde0d82ed1 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:42.759425 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7edac-1018-476a-ad15-f5fde0d82ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:42.759425 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7edac-1018-476a-ad15-f5fde0d82ed1 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:42.760821 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 591/1167] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:18:42.766279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1806745-a0d8-4d7b-b8a9-fdd9e8ddc770 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:42.766720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1806745-a0d8-4d7b-b8a9-fdd9e8ddc770 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:42.766821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1806745-a0d8-4d7b-b8a9-fdd9e8ddc770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:42.785025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a1806745-a0d8-4d7b-b8a9-fdd9e8ddc770 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:42.786193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a1806745-a0d8-4d7b-b8a9-fdd9e8ddc770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:42.852617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 577/1168] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:43.130385 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7edac-1018-476a-ad15-f5fde0d82ed1 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:43.228471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7edac-1018-476a-ad15-f5fde0d82ed1 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:43.261142 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91249f5-53f2-437b-aef9-85979dd3e06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:43.261732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91249f5-53f2-437b-aef9-85979dd3e06a None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:43.262010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91249f5-53f2-437b-aef9-85979dd3e06a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:43.262280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91249f5-53f2-437b-aef9-85979dd3e06a None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:43.262944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91249f5-53f2-437b-aef9-85979dd3e06a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:43.264320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91249f5-53f2-437b-aef9-85979dd3e06a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:43.279705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 592/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Sun Sep 11 05:18:43 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:18:45.433132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8742-b482-4b30-994b-a63633a94417 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:45.433663 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8742-b482-4b30-994b-a63633a94417 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:45.433663 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8742-b482-4b30-994b-a63633a94417 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:45.435017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 578/1170] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:45.440797 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ace8e0-531a-4129-87c0-ac2c0eacbc6d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:45.441174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ace8e0-531a-4129-87c0-ac2c0eacbc6d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:45.441346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ace8e0-531a-4129-87c0-ac2c0eacbc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:45.459774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-56ace8e0-531a-4129-87c0-ac2c0eacbc6d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:45.460990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-56ace8e0-531a-4129-87c0-ac2c0eacbc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:45.488253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:45.526431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 593/1171] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:45.802497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8742-b482-4b30-994b-a63633a94417 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:45.906371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:45.912601 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8742-b482-4b30-994b-a63633a94417 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:45.945667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251e96c-765e-4f63-86cb-ce0782d7c050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:45.946280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251e96c-765e-4f63-86cb-ce0782d7c050 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:45.946552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251e96c-765e-4f63-86cb-ce0782d7c050 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:45.946816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251e96c-765e-4f63-86cb-ce0782d7c050 None admin] PATH_INFO: `/v3/domains/695e4e62bcb349d1b70fbb0c67acff08` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:45.957447 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251e96c-765e-4f63-86cb-ce0782d7c050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=695e4e62bcb349d1b70fbb0c67acff08)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:45.958878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251e96c-765e-4f63-86cb-ce0782d7c050 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:45.960458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 579/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Sun Sep 11 05:18:45 2022] GET /identity/v3/domains/695e4e62bcb349d1b70fbb0c67acff08 => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:45.966257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ace8e0-531a-4129-87c0-ac2c0eacbc6d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:46.062575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ace8e0-531a-4129-87c0-ac2c0eacbc6d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:46.094781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72fa30-aace-463f-b650-0dd92a60fada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:46.095371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72fa30-aace-463f-b650-0dd92a60fada None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:46.095645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72fa30-aace-463f-b650-0dd92a60fada None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:46.095913 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72fa30-aace-463f-b650-0dd92a60fada None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:46.096657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72fa30-aace-463f-b650-0dd92a60fada None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:46.098068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72fa30-aace-463f-b650-0dd92a60fada None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:46.123444 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 594/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:18:45 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:18:48.290450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4ee8a-b019-414f-834e-b6204c8e6219 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:48.290979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4ee8a-b019-414f-834e-b6204c8e6219 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:48.290979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4ee8a-b019-414f-834e-b6204c8e6219 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:48.292350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 580/1174] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:48.298286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222b40d-c285-4bed-9c32-5e612b915b21 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:48.298686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222b40d-c285-4bed-9c32-5e612b915b21 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:48.298840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222b40d-c285-4bed-9c32-5e612b915b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:48.316928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c222b40d-c285-4bed-9c32-5e612b915b21 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:48.318115 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c222b40d-c285-4bed-9c32-5e612b915b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:48.331289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:48.384061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 595/1175] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:18:48.654266 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4ee8a-b019-414f-834e-b6204c8e6219 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:48.754837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4ee8a-b019-414f-834e-b6204c8e6219 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:48.787799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844faede-9c7c-40f8-8614-5d1692c264a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:48.788476 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844faede-9c7c-40f8-8614-5d1692c264a2 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:48.788758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844faede-9c7c-40f8-8614-5d1692c264a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:48.789031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844faede-9c7c-40f8-8614-5d1692c264a2 None admin] PATH_INFO: `/v3/domains/695e4e62bcb349d1b70fbb0c67acff08` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:48.790163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844faede-9c7c-40f8-8614-5d1692c264a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=695e4e62bcb349d1b70fbb0c67acff08)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:48.791607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844faede-9c7c-40f8-8614-5d1692c264a2 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:48.793206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 581/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Sun Sep 11 05:18:48 2022] GET /identity/v3/domains/695e4e62bcb349d1b70fbb0c67acff08 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:18:48.799508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222b40d-c285-4bed-9c32-5e612b915b21 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:48.894328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222b40d-c285-4bed-9c32-5e612b915b21 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:48.926689 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec3b37-42d1-4440-b395-07981a9a4127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:48.927222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3b37-42d1-4440-b395-07981a9a4127 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:48.927501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3b37-42d1-4440-b395-07981a9a4127 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:48.927772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3b37-42d1-4440-b395-07981a9a4127 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:48.928440 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec3b37-42d1-4440-b395-07981a9a4127 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:48.929852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec3b37-42d1-4440-b395-07981a9a4127 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:48.949864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 596/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:18:48 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:18:51.115136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427a94b-ab3e-426d-bf41-b467926e0e8e None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:51.115656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427a94b-ab3e-426d-bf41-b467926e0e8e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:51.115782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427a94b-ab3e-426d-bf41-b467926e0e8e None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:51.117043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 582/1178] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:51.122274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6baf31-7fee-4d36-8463-e5d820864dc1 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:51.122709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6baf31-7fee-4d36-8463-e5d820864dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:51.122814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6baf31-7fee-4d36-8463-e5d820864dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:51.130844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:51.142601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1f6baf31-7fee-4d36-8463-e5d820864dc1 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:51.143798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1f6baf31-7fee-4d36-8463-e5d820864dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:51.208461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 597/1179] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:51.476928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427a94b-ab3e-426d-bf41-b467926e0e8e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:51.505080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:51.580237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427a94b-ab3e-426d-bf41-b467926e0e8e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:51.612951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277f4e7-783b-4c58-baf7-d1f61119ae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:51.613560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277f4e7-783b-4c58-baf7-d1f61119ae33 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:51.613840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277f4e7-783b-4c58-baf7-d1f61119ae33 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:51.614111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277f4e7-783b-4c58-baf7-d1f61119ae33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:51.624932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277f4e7-783b-4c58-baf7-d1f61119ae33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c277f4e7-783b-4c58-baf7-d1f61119ae33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:18:51.629921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:51.631058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:51.632144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:51.632144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:51.632144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:51.632144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:51.632144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:18:51.632144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:18:51.632144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:18:51.632144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:18:51.632144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:18:51.632144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:18:51.632144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:51.632144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:18:51.635828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 583/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:18:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:51.636945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6baf31-7fee-4d36-8463-e5d820864dc1 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:51.731198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6baf31-7fee-4d36-8463-e5d820864dc1 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:51.765183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bf4e5-73d9-477a-9dc5-f7f38ca6cf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:51.765777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bf4e5-73d9-477a-9dc5-f7f38ca6cf87 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:51.766058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bf4e5-73d9-477a-9dc5-f7f38ca6cf87 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:51.766324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bf4e5-73d9-477a-9dc5-f7f38ca6cf87 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:51.766971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bf4e5-73d9-477a-9dc5-f7f38ca6cf87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:51.767265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bf4e5-73d9-477a-9dc5-f7f38ca6cf87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:51.768659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bf4e5-73d9-477a-9dc5-f7f38ca6cf87 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:51.769944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:51.777085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 598/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:18:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:51.783134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277f4e7-783b-4c58-baf7-d1f61119ae33 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:51.867045 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:51.882998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277f4e7-783b-4c58-baf7-d1f61119ae33 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:51.916088 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5163158-d674-4f69-9f95-923cc6ff8e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:51.916711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5163158-d674-4f69-9f95-923cc6ff8e3f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:51.916992 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5163158-d674-4f69-9f95-923cc6ff8e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:51.917263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5163158-d674-4f69-9f95-923cc6ff8e3f None admin] PATH_INFO: `/v3/users/be289a03d4a04bf3ad2a0252ccc5f8b4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:51.930586 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5163158-d674-4f69-9f95-923cc6ff8e3f None admin] RBAC: Authorizing `identity:get_user(user_id=be289a03d4a04bf3ad2a0252ccc5f8b4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:51.932042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5163158-d674-4f69-9f95-923cc6ff8e3f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:51.933626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 584/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:51 2022] GET /identity/v3/users/be289a03d4a04bf3ad2a0252ccc5f8b4 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:51.940159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bf4e5-73d9-477a-9dc5-f7f38ca6cf87 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:52.037085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bf4e5-73d9-477a-9dc5-f7f38ca6cf87 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:52.069699 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34eba2a-102f-4246-a116-43dd4d69f323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:52.070301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eba2a-102f-4246-a116-43dd4d69f323 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:52.070579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eba2a-102f-4246-a116-43dd4d69f323 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:52.070849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eba2a-102f-4246-a116-43dd4d69f323 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:52.129528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:52.136195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34eba2a-102f-4246-a116-43dd4d69f323 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:52.137638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34eba2a-102f-4246-a116-43dd4d69f323 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:52.230710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 599/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:18:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:18:52.237370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5163158-d674-4f69-9f95-923cc6ff8e3f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:52.334804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5163158-d674-4f69-9f95-923cc6ff8e3f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:52.368772 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9d9cd-25a5-4c92-995a-627d4014cd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:52.369378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9d9cd-25a5-4c92-995a-627d4014cd20 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:52.369655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9d9cd-25a5-4c92-995a-627d4014cd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:52.369923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9d9cd-25a5-4c92-995a-627d4014cd20 None admin] PATH_INFO: `/v3/projects/e25a49f113cf47ac9be263b304dedc49` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:52.371015 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9d9cd-25a5-4c92-995a-627d4014cd20 None admin] RBAC: Authorizing `identity:get_project(project_id=e25a49f113cf47ac9be263b304dedc49)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:52.372475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9d9cd-25a5-4c92-995a-627d4014cd20 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:52.374253 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 585/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:18:52 2022] GET /identity/v3/projects/e25a49f113cf47ac9be263b304dedc49 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:52.380706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34eba2a-102f-4246-a116-43dd4d69f323 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:52.472951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:52.479087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34eba2a-102f-4246-a116-43dd4d69f323 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:52.515677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb31df-d55a-460a-a065-eb444db0ac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:52.516534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb31df-d55a-460a-a065-eb444db0ac80 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:52.516821 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb31df-d55a-460a-a065-eb444db0ac80 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:52.517090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb31df-d55a-460a-a065-eb444db0ac80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:52.517800 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb31df-d55a-460a-a065-eb444db0ac80 None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=e25a49f113cf47ac9be263b304dedc49, user.id=be289a03d4a04bf3ad2a0252ccc5f8b4) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:52.518101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb31df-d55a-460a-a065-eb444db0ac80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:52.519507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb31df-d55a-460a-a065-eb444db0ac80 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:52.524630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 600/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:18:52 2022] GET /identity/v3/role_assignments?user.id=be289a03d4a04bf3ad2a0252ccc5f8b4&scope.project.id=e25a49f113cf47ac9be263b304dedc49&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:54.626187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65846d9d-46e2-4974-b6bf-77c962a166b9 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:54.626724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65846d9d-46e2-4974-b6bf-77c962a166b9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:54.626724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65846d9d-46e2-4974-b6bf-77c962a166b9 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:54.628076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 586/1186] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:54.633394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60948fe6-55b8-4ad7-bc0a-99aac3fa2f61 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:54.633829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60948fe6-55b8-4ad7-bc0a-99aac3fa2f61 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:54.633923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60948fe6-55b8-4ad7-bc0a-99aac3fa2f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:54.651726 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-60948fe6-55b8-4ad7-bc0a-99aac3fa2f61 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:54.652935 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-60948fe6-55b8-4ad7-bc0a-99aac3fa2f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:54.716958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 601/1187] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:54.987030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65846d9d-46e2-4974-b6bf-77c962a166b9 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:55.001778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:55.087401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65846d9d-46e2-4974-b6bf-77c962a166b9 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:55.121037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06a9c8-380b-4773-b2fc-9c43537435b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:55.121662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06a9c8-380b-4773-b2fc-9c43537435b3 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:55.121940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06a9c8-380b-4773-b2fc-9c43537435b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:55.122210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06a9c8-380b-4773-b2fc-9c43537435b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:55.132330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06a9c8-380b-4773-b2fc-9c43537435b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-4e06a9c8-380b-4773-b2fc-9c43537435b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:18:55.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:55.138180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:55.139052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:55.139052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:55.139052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:55.139052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:55.139052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:18:55.139052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:18:55.139052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:18:55.139052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:18:55.139052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:18:55.139052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:18:55.139052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:55.139052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:18:55.139052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 587/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:18:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:55.144281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60948fe6-55b8-4ad7-bc0a-99aac3fa2f61 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:55.229479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:55.239395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60948fe6-55b8-4ad7-bc0a-99aac3fa2f61 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:55.272211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab737ce-dfaa-4646-8ca7-1478bb823346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:55.272807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab737ce-dfaa-4646-8ca7-1478bb823346 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:55.273083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab737ce-dfaa-4646-8ca7-1478bb823346 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:55.273351 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab737ce-dfaa-4646-8ca7-1478bb823346 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:55.273997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab737ce-dfaa-4646-8ca7-1478bb823346 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:55.274293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab737ce-dfaa-4646-8ca7-1478bb823346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:55.275635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab737ce-dfaa-4646-8ca7-1478bb823346 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:55.284623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 602/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:18:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:55.290381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06a9c8-380b-4773-b2fc-9c43537435b3 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:55.360669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:55.389904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06a9c8-380b-4773-b2fc-9c43537435b3 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:55.422882 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489a5a-0d35-4fb8-a692-b122c3466ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:55.423494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489a5a-0d35-4fb8-a692-b122c3466ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:55.423773 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489a5a-0d35-4fb8-a692-b122c3466ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:55.424071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c489a5a-0d35-4fb8-a692-b122c3466ff0 None admin] PATH_INFO: `/v3/users/be289a03d4a04bf3ad2a0252ccc5f8b4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:55.425169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c489a5a-0d35-4fb8-a692-b122c3466ff0 None admin] RBAC: Authorizing `identity:get_user(user_id=be289a03d4a04bf3ad2a0252ccc5f8b4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:55.426597 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c489a5a-0d35-4fb8-a692-b122c3466ff0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:55.428171 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 588/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:55 2022] GET /identity/v3/users/be289a03d4a04bf3ad2a0252ccc5f8b4 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:55.433907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab737ce-dfaa-4646-8ca7-1478bb823346 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:55.529044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab737ce-dfaa-4646-8ca7-1478bb823346 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:55.561846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b82993-f578-4b20-b207-341eb197c4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:55.562498 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b82993-f578-4b20-b207-341eb197c4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:55.562779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b82993-f578-4b20-b207-341eb197c4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:55.563049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b82993-f578-4b20-b207-341eb197c4f8 None admin] PATH_INFO: `/v3/projects/e25a49f113cf47ac9be263b304dedc49` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:55.564185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b82993-f578-4b20-b207-341eb197c4f8 None admin] RBAC: Authorizing `identity:get_project(project_id=e25a49f113cf47ac9be263b304dedc49)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:55.565602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b82993-f578-4b20-b207-341eb197c4f8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:55.567222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 603/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:18:55 2022] GET /identity/v3/projects/e25a49f113cf47ac9be263b304dedc49 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:55.573082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489a5a-0d35-4fb8-a692-b122c3466ff0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:55.672320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c489a5a-0d35-4fb8-a692-b122c3466ff0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:55.704745 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb4efb-b0ae-4c8e-a376-20b848d81fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:55.705385 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb4efb-b0ae-4c8e-a376-20b848d81fb0 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:55.705665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb4efb-b0ae-4c8e-a376-20b848d81fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:55.705938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb4efb-b0ae-4c8e-a376-20b848d81fb0 None admin] PATH_INFO: `/v3/projects/e25a49f113cf47ac9be263b304dedc49/users/be289a03d4a04bf3ad2a0252ccc5f8b4/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:55.707823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb4efb-b0ae-4c8e-a376-20b848d81fb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e25a49f113cf47ac9be263b304dedc49, user_id=be289a03d4a04bf3ad2a0252ccc5f8b4, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:55.709268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb4efb-b0ae-4c8e-a376-20b848d81fb0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:55.718213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:55.722952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 589/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:18:55 2022] PUT /identity/v3/projects/e25a49f113cf47ac9be263b304dedc49/users/be289a03d4a04bf3ad2a0252ccc5f8b4/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:18:57.836115 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2effe41-63ce-4bec-ab96-b7bff0e304c7 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:57.836604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2effe41-63ce-4bec-ab96-b7bff0e304c7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:57.836701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2effe41-63ce-4bec-ab96-b7bff0e304c7 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:57.838026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 604/1193] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:18:57.843866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf6897-0ec7-4268-a029-399acee7a9ca None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:57.844330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf6897-0ec7-4268-a029-399acee7a9ca None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:57.844523 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf6897-0ec7-4268-a029-399acee7a9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:57.862342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d5cf6897-0ec7-4268-a029-399acee7a9ca None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:18:57.863528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d5cf6897-0ec7-4268-a029-399acee7a9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:18:57.929919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 590/1194] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:18:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:18:58.204447 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2effe41-63ce-4bec-ab96-b7bff0e304c7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:58.226457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:58.303370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2effe41-63ce-4bec-ab96-b7bff0e304c7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:58.335107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b90df0-96d5-4fbb-a3a7-a30721fac683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:58.335721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b90df0-96d5-4fbb-a3a7-a30721fac683 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:58.336019 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b90df0-96d5-4fbb-a3a7-a30721fac683 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:58.336290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b90df0-96d5-4fbb-a3a7-a30721fac683 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:58.347299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b90df0-96d5-4fbb-a3a7-a30721fac683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-21b90df0-96d5-4fbb-a3a7-a30721fac683 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:18:58.351977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:18:58.353075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:18:58.354077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:18:58.354077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:18:58.354077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:18:58.354077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:18:58.354077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:18:58.354077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:18:58.354077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:18:58.354077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:18:58.354077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:18:58.354077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:18:58.354077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:18:58.354077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:18:58.354077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 605/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:18:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:18:58.359045 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf6897-0ec7-4268-a029-399acee7a9ca None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:58.458822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf6897-0ec7-4268-a029-399acee7a9ca None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:58.486609 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:58.492406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b1199-a242-4838-83b3-fb1595e90a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:58.492996 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b1199-a242-4838-83b3-fb1595e90a45 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:58.493273 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b1199-a242-4838-83b3-fb1595e90a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:58.493541 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b1199-a242-4838-83b3-fb1595e90a45 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:58.494190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b1199-a242-4838-83b3-fb1595e90a45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:58.494489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b1199-a242-4838-83b3-fb1595e90a45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:58.495855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b1199-a242-4838-83b3-fb1595e90a45 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:58.503828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 591/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:18:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:58.509958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b90df0-96d5-4fbb-a3a7-a30721fac683 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:58.587024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:58.605941 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b90df0-96d5-4fbb-a3a7-a30721fac683 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:58.637493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b1d42-be6d-4575-a91d-940d71ee2360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:58.638108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b1d42-be6d-4575-a91d-940d71ee2360 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:58.638383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b1d42-be6d-4575-a91d-940d71ee2360 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:58.638649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b1d42-be6d-4575-a91d-940d71ee2360 None admin] PATH_INFO: `/v3/users/be289a03d4a04bf3ad2a0252ccc5f8b4` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:58.640408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b1d42-be6d-4575-a91d-940d71ee2360 None admin] RBAC: Authorizing `identity:get_user(user_id=be289a03d4a04bf3ad2a0252ccc5f8b4)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:58.641826 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b1d42-be6d-4575-a91d-940d71ee2360 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:58.643367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 606/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Sun Sep 11 05:18:58 2022] GET /identity/v3/users/be289a03d4a04bf3ad2a0252ccc5f8b4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:58.649028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b1199-a242-4838-83b3-fb1595e90a45 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:58.747495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b1199-a242-4838-83b3-fb1595e90a45 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:58.780924 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e619a8-a423-4bc8-86d2-494faf38cfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:58.781512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e619a8-a423-4bc8-86d2-494faf38cfb5 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:58.781791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e619a8-a423-4bc8-86d2-494faf38cfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:58.782060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e619a8-a423-4bc8-86d2-494faf38cfb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:58.838916 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:58.848699 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e619a8-a423-4bc8-86d2-494faf38cfb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:58.850145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e619a8-a423-4bc8-86d2-494faf38cfb5 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:58.945425 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 592/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:18:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:18:58.951492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b1d42-be6d-4575-a91d-940d71ee2360 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:59.047602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b1d42-be6d-4575-a91d-940d71ee2360 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:59.079988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2693682-8bb6-4784-8f25-1b49bd83644f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:59.080631 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2693682-8bb6-4784-8f25-1b49bd83644f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:59.080911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2693682-8bb6-4784-8f25-1b49bd83644f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:59.081183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2693682-8bb6-4784-8f25-1b49bd83644f None admin] PATH_INFO: `/v3/projects/e25a49f113cf47ac9be263b304dedc49` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:59.082287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2693682-8bb6-4784-8f25-1b49bd83644f None admin] RBAC: Authorizing `identity:get_project(project_id=e25a49f113cf47ac9be263b304dedc49)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:59.083704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2693682-8bb6-4784-8f25-1b49bd83644f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:59.085335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 607/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Sun Sep 11 05:18:58 2022] GET /identity/v3/projects/e25a49f113cf47ac9be263b304dedc49 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:18:59.091387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e619a8-a423-4bc8-86d2-494faf38cfb5 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:18:59.179303 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:18:59.189266 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e619a8-a423-4bc8-86d2-494faf38cfb5 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:18:59.222242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6fd58-ce8e-4279-8ad3-3510453885cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:18:59.222828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6fd58-ce8e-4279-8ad3-3510453885cd None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:18:59.223104 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6fd58-ce8e-4279-8ad3-3510453885cd None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:18:59.223372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6fd58-ce8e-4279-8ad3-3510453885cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:18:59.224106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd6fd58-ce8e-4279-8ad3-3510453885cd None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=e25a49f113cf47ac9be263b304dedc49, user.id=be289a03d4a04bf3ad2a0252ccc5f8b4) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:18:59.224410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd6fd58-ce8e-4279-8ad3-3510453885cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:18:59.225828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd6fd58-ce8e-4279-8ad3-3510453885cd None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:18:59.231375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 593/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:18:59 2022] GET /identity/v3/role_assignments?user.id=be289a03d4a04bf3ad2a0252ccc5f8b4&scope.project.id=e25a49f113cf47ac9be263b304dedc49&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:19:09.075638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:19:09.228863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=22 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:19:54.805119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb684add-81b4-4a09-a80e-963f6e4bacf8 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:54.805119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb684add-81b4-4a09-a80e-963f6e4bacf8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:54.805119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb684add-81b4-4a09-a80e-963f6e4bacf8 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:54.806102 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 608/1201] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:19:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:19:54.814336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a95e8d-caf8-46ea-a8bc-d4fc8328f4d6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:54.814833 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a95e8d-caf8-46ea-a8bc-d4fc8328f4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:54.814942 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a95e8d-caf8-46ea-a8bc-d4fc8328f4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:54.842245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-32a95e8d-caf8-46ea-a8bc-d4fc8328f4d6 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:19:54.844062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-32a95e8d-caf8-46ea-a8bc-d4fc8328f4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:19:54.936176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 594/1202] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:19:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:19:55.206522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb684add-81b4-4a09-a80e-963f6e4bacf8 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:19:55.339303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb684add-81b4-4a09-a80e-963f6e4bacf8 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:19:55.386313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da71e2-78e6-462d-a0cf-ddbe99ccbaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:19:55.388056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da71e2-78e6-462d-a0cf-ddbe99ccbaef None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:55.388578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da71e2-78e6-462d-a0cf-ddbe99ccbaef None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:55.389139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da71e2-78e6-462d-a0cf-ddbe99ccbaef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:55.403320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da71e2-78e6-462d-a0cf-ddbe99ccbaef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:19:55.405416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da71e2-78e6-462d-a0cf-ddbe99ccbaef None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-e0da71e2-78e6-462d-a0cf-ddbe99ccbaef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:19:55.421314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:19:55.423133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:19:55.424197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:19:55.424197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:19:55.424197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:19:55.424197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 609/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:19:55.429829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a95e8d-caf8-46ea-a8bc-d4fc8328f4d6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:19:55.550417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a95e8d-caf8-46ea-a8bc-d4fc8328f4d6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:19:55.585642 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:19:55.593762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1eb66b-6a04-4e36-a456-b403e1e5597b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:19:55.594748 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eb66b-6a04-4e36-a456-b403e1e5597b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:55.595222 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eb66b-6a04-4e36-a456-b403e1e5597b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:55.595801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eb66b-6a04-4e36-a456-b403e1e5597b None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:55.596713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1eb66b-6a04-4e36-a456-b403e1e5597b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:19:55.597238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1eb66b-6a04-4e36-a456-b403e1e5597b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:19:55.598956 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1eb66b-6a04-4e36-a456-b403e1e5597b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:19:55.617900 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 595/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:19:55.619256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da71e2-78e6-462d-a0cf-ddbe99ccbaef None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:19:55.695410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:19:55.759850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da71e2-78e6-462d-a0cf-ddbe99ccbaef None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:19:55.804895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0f4b0-caa9-49e4-a5bc-8f2b93ee0be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:19:55.805470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0f4b0-caa9-49e4-a5bc-8f2b93ee0be5 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:55.805999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0f4b0-caa9-49e4-a5bc-8f2b93ee0be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:55.805999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0f4b0-caa9-49e4-a5bc-8f2b93ee0be5 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:55.806640 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0f4b0-caa9-49e4-a5bc-8f2b93ee0be5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:19:55.808191 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0f4b0-caa9-49e4-a5bc-8f2b93ee0be5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:19:55.835882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 610/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Sun Sep 11 05:19:55 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:19:58.219498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504361bf-f58b-4ea8-9582-4cf4b6ec31e9 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:58.220096 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504361bf-f58b-4ea8-9582-4cf4b6ec31e9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:58.220096 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504361bf-f58b-4ea8-9582-4cf4b6ec31e9 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:58.221568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 596/1206] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:19:58.226881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a3c9c-3a92-4e66-94e8-f0f1415956c9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:58.227315 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a3c9c-3a92-4e66-94e8-f0f1415956c9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:58.227416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a3c9c-3a92-4e66-94e8-f0f1415956c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:58.250548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bf0a3c9c-3a92-4e66-94e8-f0f1415956c9 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:19:58.250548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bf0a3c9c-3a92-4e66-94e8-f0f1415956c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:19:58.334245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 611/1207] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:19:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:19:58.607583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504361bf-f58b-4ea8-9582-4cf4b6ec31e9 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:19:58.741852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504361bf-f58b-4ea8-9582-4cf4b6ec31e9 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:19:58.785444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b55094-96f1-4e5c-b425-8b4861fdb5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:19:58.786488 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b55094-96f1-4e5c-b425-8b4861fdb5cf None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:58.787102 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b55094-96f1-4e5c-b425-8b4861fdb5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:58.787378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b55094-96f1-4e5c-b425-8b4861fdb5cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:58.803543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b55094-96f1-4e5c-b425-8b4861fdb5cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-58b55094-96f1-4e5c-b425-8b4861fdb5cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:19:58.808758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:19:58.811352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:19:58.812558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:19:58.812558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:19:58.812558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:19:58.812558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:19:58.812558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:19:58.812558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:19:58.812558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:19:58.812558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:19:58.812558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:19:58.812558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:19:58.812558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:19:58.812558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:19:58.812558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 597/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:19:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:19:58.816984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a3c9c-3a92-4e66-94e8-f0f1415956c9 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:19:58.860265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:19:58.928105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a3c9c-3a92-4e66-94e8-f0f1415956c9 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:19:58.967158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb2aa0-d1a8-4e5d-9adf-f4f91e2d5570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:19:58.967776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb2aa0-d1a8-4e5d-9adf-f4f91e2d5570 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:58.968083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb2aa0-d1a8-4e5d-9adf-f4f91e2d5570 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:58.968386 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb2aa0-d1a8-4e5d-9adf-f4f91e2d5570 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:58.969085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb2aa0-d1a8-4e5d-9adf-f4f91e2d5570 None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:19:58.969436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb2aa0-d1a8-4e5d-9adf-f4f91e2d5570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:19:58.970754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb2aa0-d1a8-4e5d-9adf-f4f91e2d5570 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:19:58.979075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 612/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:19:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:19:58.985894 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b55094-96f1-4e5c-b425-8b4861fdb5cf None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:19:59.045488 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:19:59.111845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b55094-96f1-4e5c-b425-8b4861fdb5cf None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:19:59.158299 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1256e4-cd03-48fe-8ffe-4232ed77a4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:19:59.159381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1256e4-cd03-48fe-8ffe-4232ed77a4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:59.159495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1256e4-cd03-48fe-8ffe-4232ed77a4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:59.159705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1256e4-cd03-48fe-8ffe-4232ed77a4a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:59.172403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1256e4-cd03-48fe-8ffe-4232ed77a4a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:19:59.174375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1256e4-cd03-48fe-8ffe-4232ed77a4a0 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-8d1256e4-cd03-48fe-8ffe-4232ed77a4a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:19:59.190963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:19:59.192233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:19:59.193405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:19:59.193405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:19:59.193405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:19:59.194097 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 598/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:19:59.200993 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb2aa0-d1a8-4e5d-9adf-f4f91e2d5570 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:19:59.316626 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:19:59.320255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb2aa0-d1a8-4e5d-9adf-f4f91e2d5570 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:19:59.364220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f048d-2b5e-49bb-a9b4-9521d8e07c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:19:59.364434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f048d-2b5e-49bb-a9b4-9521d8e07c70 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:59.364607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f048d-2b5e-49bb-a9b4-9521d8e07c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:59.364926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f048d-2b5e-49bb-a9b4-9521d8e07c70 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:59.365812 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f048d-2b5e-49bb-a9b4-9521d8e07c70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:19:59.366157 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f048d-2b5e-49bb-a9b4-9521d8e07c70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:19:59.368097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f048d-2b5e-49bb-a9b4-9521d8e07c70 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:19:59.386611 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 613/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:19:59.392796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1256e4-cd03-48fe-8ffe-4232ed77a4a0 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:19:59.520279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1256e4-cd03-48fe-8ffe-4232ed77a4a0 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:19:59.522435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:19:59.582632 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af914e37-d180-4df5-99af-a662dd015cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:19:59.583501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af914e37-d180-4df5-99af-a662dd015cde None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:59.583950 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af914e37-d180-4df5-99af-a662dd015cde None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:59.585873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af914e37-d180-4df5-99af-a662dd015cde None admin] PATH_INFO: `/v3/users/placement` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:59.595226 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af914e37-d180-4df5-99af-a662dd015cde None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:19:59.596853 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af914e37-d180-4df5-99af-a662dd015cde None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-af914e37-d180-4df5-99af-a662dd015cde None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:19:59.608876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:19:59.610091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:19:59.611238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:19:59.611238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:19:59.611238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:19:59.611238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:19:59.611238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:19:59.611238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:19:59.611238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:19:59.611238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:19:59.611238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 05:19:59.611238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:19:59.611964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 599/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:19:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:19:59.618213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f048d-2b5e-49bb-a9b4-9521d8e07c70 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:19:59.731468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f048d-2b5e-49bb-a9b4-9521d8e07c70 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:19:59.766784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35e545-90d3-46d2-b6a1-8178567d922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:19:59.767395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35e545-90d3-46d2-b6a1-8178567d922a None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:59.767676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35e545-90d3-46d2-b6a1-8178567d922a None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:59.767946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35e545-90d3-46d2-b6a1-8178567d922a None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:59.768695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35e545-90d3-46d2-b6a1-8178567d922a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:19:59.769001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35e545-90d3-46d2-b6a1-8178567d922a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:19:59.770424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35e545-90d3-46d2-b6a1-8178567d922a None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:19:59.791879 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 614/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 11 05:19:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:19:59.801094 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af914e37-d180-4df5-99af-a662dd015cde None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:19:59.916647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af914e37-d180-4df5-99af-a662dd015cde None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:19:59.955894 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a96268-2f7d-4c19-84f1-27b7a13d4e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:19:59.956537 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a96268-2f7d-4c19-84f1-27b7a13d4e4d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:19:59.956816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a96268-2f7d-4c19-84f1-27b7a13d4e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:19:59.957093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a96268-2f7d-4c19-84f1-27b7a13d4e4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:19:59.994142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:00.042943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a96268-2f7d-4c19-84f1-27b7a13d4e4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:00.044989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a96268-2f7d-4c19-84f1-27b7a13d4e4d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:00.163375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 600/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:19:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:20:00.170229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35e545-90d3-46d2-b6a1-8178567d922a None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:00.192485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:00.292140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35e545-90d3-46d2-b6a1-8178567d922a None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:00.333183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eabec5-28a4-45df-ac12-2980cb6aa771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:00.333928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eabec5-28a4-45df-ac12-2980cb6aa771 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:00.334454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eabec5-28a4-45df-ac12-2980cb6aa771 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:00.334800 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eabec5-28a4-45df-ac12-2980cb6aa771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:00.347285 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eabec5-28a4-45df-ac12-2980cb6aa771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:00.349166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eabec5-28a4-45df-ac12-2980cb6aa771 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-f7eabec5-28a4-45df-ac12-2980cb6aa771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:20:00.368670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:00.370113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:20:00.371775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:20:00.371775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:00.371775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:00.371775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 615/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:00.380438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a96268-2f7d-4c19-84f1-27b7a13d4e4d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:00.448212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:00.496254 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a96268-2f7d-4c19-84f1-27b7a13d4e4d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:00.544742 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3848c47-444b-4df4-b6ff-e18426dd80a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:00.545433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3848c47-444b-4df4-b6ff-e18426dd80a9 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:00.545718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3848c47-444b-4df4-b6ff-e18426dd80a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:00.545992 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3848c47-444b-4df4-b6ff-e18426dd80a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:00.551354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3848c47-444b-4df4-b6ff-e18426dd80a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:00.551354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3848c47-444b-4df4-b6ff-e18426dd80a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:00.551354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3848c47-444b-4df4-b6ff-e18426dd80a9 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:00.562880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 601/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:20:00.569149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eabec5-28a4-45df-ac12-2980cb6aa771 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:00.657194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:00.699525 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eabec5-28a4-45df-ac12-2980cb6aa771 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:00.740926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37144090-0b06-4185-a7cb-9622549ab9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:00.741591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37144090-0b06-4185-a7cb-9622549ab9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:00.741870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37144090-0b06-4185-a7cb-9622549ab9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:00.742142 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37144090-0b06-4185-a7cb-9622549ab9a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:00.749268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37144090-0b06-4185-a7cb-9622549ab9a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:00.750894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37144090-0b06-4185-a7cb-9622549ab9a7 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-37144090-0b06-4185-a7cb-9622549ab9a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:20:00.763408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:00.764746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:00.765762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:20:00.765762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:20:00.765762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:20:00.765762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:20:00.765762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:20:00.765762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:20:00.765762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:20:00.765762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:00.765762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 616/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:20:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:20:00.771351 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3848c47-444b-4df4-b6ff-e18426dd80a9 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:00.886693 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3848c47-444b-4df4-b6ff-e18426dd80a9 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:00.913854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:00.928273 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82938595-1e56-426a-95b7-9d5599795e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:00.928919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82938595-1e56-426a-95b7-9d5599795e9b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:00.929255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82938595-1e56-426a-95b7-9d5599795e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:00.929568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82938595-1e56-426a-95b7-9d5599795e9b None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:00.930359 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82938595-1e56-426a-95b7-9d5599795e9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:00.930678 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82938595-1e56-426a-95b7-9d5599795e9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:00.932255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82938595-1e56-426a-95b7-9d5599795e9b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:00.945821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 602/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:20:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:00.953594 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37144090-0b06-4185-a7cb-9622549ab9a7 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:01.068814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37144090-0b06-4185-a7cb-9622549ab9a7 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:01.099454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:01.105592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60799ba-ab2f-46b4-824f-8a128481ad0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:01.106444 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60799ba-ab2f-46b4-824f-8a128481ad0c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:01.106721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60799ba-ab2f-46b4-824f-8a128481ad0c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:01.106989 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60799ba-ab2f-46b4-824f-8a128481ad0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:01.107710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60799ba-ab2f-46b4-824f-8a128481ad0c None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=4e985df8ad6c4bc7b3cd1328924e3ce2) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:01.108040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60799ba-ab2f-46b4-824f-8a128481ad0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:01.109571 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60799ba-ab2f-46b4-824f-8a128481ad0c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:01.115430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 617/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:20:00 2022] GET /identity/v3/role_assignments?user.id=4e985df8ad6c4bc7b3cd1328924e3ce2&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:20:03.462792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eab96-d249-4701-bf42-81cd07a8d829 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:03.463352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eab96-d249-4701-bf42-81cd07a8d829 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:03.463352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eab96-d249-4701-bf42-81cd07a8d829 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:03.464882 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 603/1220] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:20:03.470026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d4323-a16f-4f65-9944-c549a71ac81e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:03.470530 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d4323-a16f-4f65-9944-c549a71ac81e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:03.470635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d4323-a16f-4f65-9944-c549a71ac81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:03.489274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5c6d4323-a16f-4f65-9944-c549a71ac81e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:20:03.490459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5c6d4323-a16f-4f65-9944-c549a71ac81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:20:03.565406 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 618/1221] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:20:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:20:03.851246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eab96-d249-4701-bf42-81cd07a8d829 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:03.963834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eab96-d249-4701-bf42-81cd07a8d829 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:04.000125 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d516f-c17d-4c84-a02b-b834d3334bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:04.000760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d516f-c17d-4c84-a02b-b834d3334bbb None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:04.001040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d516f-c17d-4c84-a02b-b834d3334bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:04.001314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d516f-c17d-4c84-a02b-b834d3334bbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:04.013513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d516f-c17d-4c84-a02b-b834d3334bbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-769d516f-c17d-4c84-a02b-b834d3334bbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:20:04.019565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:04.020934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:04.021990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:04.021990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:04.021990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:04.021990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:04.021990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:20:04.021990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:20:04.021990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:20:04.021990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:20:04.021990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:20:04.021990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:20:04.021990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:20:04.021990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:04.021990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 604/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:20:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:04.027391 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d4323-a16f-4f65-9944-c549a71ac81e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:04.134214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d4323-a16f-4f65-9944-c549a71ac81e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:04.160761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:04.172120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6c894-e1e0-44de-9d8f-ee0ccafa531b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:04.172746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6c894-e1e0-44de-9d8f-ee0ccafa531b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:04.173029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6c894-e1e0-44de-9d8f-ee0ccafa531b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:04.173309 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6c894-e1e0-44de-9d8f-ee0ccafa531b None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:04.173967 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6c894-e1e0-44de-9d8f-ee0ccafa531b None admin] RBAC: Adding query filter params (name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:04.174272 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6c894-e1e0-44de-9d8f-ee0ccafa531b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:04.175634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6c894-e1e0-44de-9d8f-ee0ccafa531b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:04.184077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 619/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:20:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:04.189932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d516f-c17d-4c84-a02b-b834d3334bbb None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:04.241081 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:04.309656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d516f-c17d-4c84-a02b-b834d3334bbb None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:04.352079 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43f92b-b78b-4287-8adf-b547aadb7eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:04.352845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43f92b-b78b-4287-8adf-b547aadb7eac None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:04.353251 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43f92b-b78b-4287-8adf-b547aadb7eac None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:04.353597 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43f92b-b78b-4287-8adf-b547aadb7eac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:04.369331 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43f92b-b78b-4287-8adf-b547aadb7eac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:04.372365 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43f92b-b78b-4287-8adf-b547aadb7eac None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-fb43f92b-b78b-4287-8adf-b547aadb7eac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:20:04.395831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:04.397401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:20:04.398416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:20:04.398416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:04.398416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:04.398416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 605/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:04.403232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6c894-e1e0-44de-9d8f-ee0ccafa531b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:04.512292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6c894-e1e0-44de-9d8f-ee0ccafa531b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:04.549699 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f9ccc-dcbb-450a-ace2-4e0da4c27e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:04.550338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f9ccc-dcbb-450a-ace2-4e0da4c27e57 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:04.550624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f9ccc-dcbb-450a-ace2-4e0da4c27e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:04.550886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f9ccc-dcbb-450a-ace2-4e0da4c27e57 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:04.551518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f9ccc-dcbb-450a-ace2-4e0da4c27e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:04.551820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f9ccc-dcbb-450a-ace2-4e0da4c27e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:04.553312 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f9ccc-dcbb-450a-ace2-4e0da4c27e57 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:04.565617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 620/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:20:04.571378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43f92b-b78b-4287-8adf-b547aadb7eac None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:04.679224 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:04.688962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43f92b-b78b-4287-8adf-b547aadb7eac None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:04.730038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb561c-721f-42f7-9c53-ccb2824d8a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:04.730948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb561c-721f-42f7-9c53-ccb2824d8a07 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:04.731150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb561c-721f-42f7-9c53-ccb2824d8a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:04.731314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb561c-721f-42f7-9c53-ccb2824d8a07 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:04.739095 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb561c-721f-42f7-9c53-ccb2824d8a07 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:04.740587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb561c-721f-42f7-9c53-ccb2824d8a07 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-70bb561c-721f-42f7-9c53-ccb2824d8a07 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:20:04.755173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:04.756204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:04.757227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:04.757227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:04.757227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:20:04.757227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:20:04.757227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:20:04.757227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:20:04.757227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:20:04.757227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:20:04.757227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 05:20:04.757227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:04.757782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 606/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:04.763647 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f9ccc-dcbb-450a-ace2-4e0da4c27e57 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:04.818557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:04.878488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f9ccc-dcbb-450a-ace2-4e0da4c27e57 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:04.920539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4ab56-39f9-47b7-802e-c6f33c9d2003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:04.921153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4ab56-39f9-47b7-802e-c6f33c9d2003 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:04.921438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4ab56-39f9-47b7-802e-c6f33c9d2003 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:04.921709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4ab56-39f9-47b7-802e-c6f33c9d2003 None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:04.922441 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4ab56-39f9-47b7-802e-c6f33c9d2003 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:04.922743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4ab56-39f9-47b7-802e-c6f33c9d2003 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:04.924225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4ab56-39f9-47b7-802e-c6f33c9d2003 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:04.942812 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 621/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 11 05:20:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:04.950934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb561c-721f-42f7-9c53-ccb2824d8a07 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:05.052444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb561c-721f-42f7-9c53-ccb2824d8a07 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:05.093733 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a3078-ab16-4adc-8a50-15bd08ed7d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:05.094571 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a3078-ab16-4adc-8a50-15bd08ed7d49 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:05.094992 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a3078-ab16-4adc-8a50-15bd08ed7d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:05.095394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a3078-ab16-4adc-8a50-15bd08ed7d49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:05.109927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a3078-ab16-4adc-8a50-15bd08ed7d49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:05.111688 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a3078-ab16-4adc-8a50-15bd08ed7d49 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-7d0a3078-ab16-4adc-8a50-15bd08ed7d49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:20:05.127010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:05.128402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:20:05.129952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:20:05.129952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:05.129952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:05.129952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 607/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:05.135112 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4ab56-39f9-47b7-802e-c6f33c9d2003 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:05.241299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4ab56-39f9-47b7-802e-c6f33c9d2003 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:05.243533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:05.275872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec970cd-f0b3-450e-ae08-daee8a98e029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:05.276501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec970cd-f0b3-450e-ae08-daee8a98e029 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:05.276777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec970cd-f0b3-450e-ae08-daee8a98e029 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:05.277050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec970cd-f0b3-450e-ae08-daee8a98e029 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:05.278228 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec970cd-f0b3-450e-ae08-daee8a98e029 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:05.278752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec970cd-f0b3-450e-ae08-daee8a98e029 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:05.280307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec970cd-f0b3-450e-ae08-daee8a98e029 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:05.295232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 622/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:05.301897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a3078-ab16-4adc-8a50-15bd08ed7d49 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:05.313482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:05.423409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a3078-ab16-4adc-8a50-15bd08ed7d49 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:05.460101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db016dfb-f9a1-4a54-8664-64a80d70ae03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:05.460724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db016dfb-f9a1-4a54-8664-64a80d70ae03 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:05.460999 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db016dfb-f9a1-4a54-8664-64a80d70ae03 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:05.461267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db016dfb-f9a1-4a54-8664-64a80d70ae03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:05.468871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db016dfb-f9a1-4a54-8664-64a80d70ae03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:05.470320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db016dfb-f9a1-4a54-8664-64a80d70ae03 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-db016dfb-f9a1-4a54-8664-64a80d70ae03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:20:05.483971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:05.485163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:05.486243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:20:05.486243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:20:05.486243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:20:05.486243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:20:05.486243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:20:05.486243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:20:05.486243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:20:05.486243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:05.486243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 608/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:20:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:05.493088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec970cd-f0b3-450e-ae08-daee8a98e029 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:05.602543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec970cd-f0b3-450e-ae08-daee8a98e029 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:05.635944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452f8e5-77fb-4900-864e-05101018a99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:05.636579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452f8e5-77fb-4900-864e-05101018a99c None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:05.636857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452f8e5-77fb-4900-864e-05101018a99c None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:05.637651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452f8e5-77fb-4900-864e-05101018a99c None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:05.638486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452f8e5-77fb-4900-864e-05101018a99c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:05.638823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452f8e5-77fb-4900-864e-05101018a99c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:05.640298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452f8e5-77fb-4900-864e-05101018a99c None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:05.653478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 623/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:20:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:05.659253 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db016dfb-f9a1-4a54-8664-64a80d70ae03 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:05.739050 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:05.779893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db016dfb-f9a1-4a54-8664-64a80d70ae03 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:05.820204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a66f5-0ba2-4b89-8cba-7b80f27d2b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:05.820881 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a66f5-0ba2-4b89-8cba-7b80f27d2b25 None admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:05.821153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a66f5-0ba2-4b89-8cba-7b80f27d2b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:05.822403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a66f5-0ba2-4b89-8cba-7b80f27d2b25 None admin] PATH_INFO: `/v3/projects/092dd30ff8b541e2a45947504831998a/users/4e985df8ad6c4bc7b3cd1328924e3ce2/roles/7a06d87f01d940b38065f296fa63d0f7` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:05.837891 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a66f5-0ba2-4b89-8cba-7b80f27d2b25 None admin] RBAC: Authorizing `identity:create_grant(project_id=092dd30ff8b541e2a45947504831998a, user_id=4e985df8ad6c4bc7b3cd1328924e3ce2, role_id=7a06d87f01d940b38065f296fa63d0f7)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:05.839494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a66f5-0ba2-4b89-8cba-7b80f27d2b25 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:05.852032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 609/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:20:05 2022] PUT /identity/v3/projects/092dd30ff8b541e2a45947504831998a/users/4e985df8ad6c4bc7b3cd1328924e3ce2/roles/7a06d87f01d940b38065f296fa63d0f7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:20:08.160519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021650d5-1e5a-47dd-812c-b1b5a490dfaf None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:08.161170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021650d5-1e5a-47dd-812c-b1b5a490dfaf None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:08.161170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021650d5-1e5a-47dd-812c-b1b5a490dfaf None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:08.162405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 624/1233] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:20:08.167700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393f5c1-d4c9-408f-a07f-4902eb44320f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:08.168248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393f5c1-d4c9-408f-a07f-4902eb44320f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:08.168949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393f5c1-d4c9-408f-a07f-4902eb44320f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:08.188410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5393f5c1-d4c9-408f-a07f-4902eb44320f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:20:08.189617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5393f5c1-d4c9-408f-a07f-4902eb44320f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:20:08.264170 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 610/1234] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:20:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:20:08.556536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021650d5-1e5a-47dd-812c-b1b5a490dfaf None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:08.583480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:08.664160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021650d5-1e5a-47dd-812c-b1b5a490dfaf None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:08.700469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039c3c1-4e92-4be9-8a09-34227ff5160e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:08.701427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039c3c1-4e92-4be9-8a09-34227ff5160e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:08.701868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039c3c1-4e92-4be9-8a09-34227ff5160e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:08.702408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039c3c1-4e92-4be9-8a09-34227ff5160e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:08.716338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039c3c1-4e92-4be9-8a09-34227ff5160e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-4039c3c1-4e92-4be9-8a09-34227ff5160e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:20:08.721892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:08.723176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:08.724261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:08.724261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:08.724261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:08.724261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:08.724261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:20:08.724261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:20:08.724261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:20:08.724261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:20:08.724261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:20:08.724261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:20:08.724261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 05:20:08.724261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:08.724261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 625/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Sun Sep 11 05:20:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 05:20:08.729338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393f5c1-d4c9-408f-a07f-4902eb44320f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:08.784053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:08.846469 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393f5c1-d4c9-408f-a07f-4902eb44320f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:08.884454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff0dba-c85f-4865-b7ce-fc601a157842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:08.885154 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff0dba-c85f-4865-b7ce-fc601a157842 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:08.885431 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff0dba-c85f-4865-b7ce-fc601a157842 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:08.885795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff0dba-c85f-4865-b7ce-fc601a157842 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:08.886547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff0dba-c85f-4865-b7ce-fc601a157842 None admin] RBAC: Adding query filter params (name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:08.886908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff0dba-c85f-4865-b7ce-fc601a157842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:08.889200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff0dba-c85f-4865-b7ce-fc601a157842 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:08.898604 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 611/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Sun Sep 11 05:20:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:08.904641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039c3c1-4e92-4be9-8a09-34227ff5160e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:09.003211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:09.022266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039c3c1-4e92-4be9-8a09-34227ff5160e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:09.061335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe37aaf-68b0-4a5c-a647-429977bc4d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:09.062151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe37aaf-68b0-4a5c-a647-429977bc4d6b None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:09.062500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe37aaf-68b0-4a5c-a647-429977bc4d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:09.062856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe37aaf-68b0-4a5c-a647-429977bc4d6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:09.075027 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe37aaf-68b0-4a5c-a647-429977bc4d6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:09.076565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe37aaf-68b0-4a5c-a647-429977bc4d6b None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-ffe37aaf-68b0-4a5c-a647-429977bc4d6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:20:09.091763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:09.093037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:20:09.094028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:20:09.094028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:09.094028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:09.094028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 626/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:09.099339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff0dba-c85f-4865-b7ce-fc601a157842 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:09.210843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff0dba-c85f-4865-b7ce-fc601a157842 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:09.223723 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:09.256018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a685a12-84db-4922-8f68-0fa4d9bc70e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:09.256634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a685a12-84db-4922-8f68-0fa4d9bc70e6 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:09.256917 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a685a12-84db-4922-8f68-0fa4d9bc70e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:09.257190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a685a12-84db-4922-8f68-0fa4d9bc70e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:09.257835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a685a12-84db-4922-8f68-0fa4d9bc70e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:09.258138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a685a12-84db-4922-8f68-0fa4d9bc70e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:09.259511 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a685a12-84db-4922-8f68-0fa4d9bc70e6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:09.271710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 612/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:09.278217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe37aaf-68b0-4a5c-a647-429977bc4d6b None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:09.388554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe37aaf-68b0-4a5c-a647-429977bc4d6b None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:09.424883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67dd00-db96-4388-91bd-25e6e7a6f8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:09.425492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67dd00-db96-4388-91bd-25e6e7a6f8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:09.425788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67dd00-db96-4388-91bd-25e6e7a6f8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:09.426037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67dd00-db96-4388-91bd-25e6e7a6f8b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:09.428840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:09.435418 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67dd00-db96-4388-91bd-25e6e7a6f8b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:09.437125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67dd00-db96-4388-91bd-25e6e7a6f8b3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-6a67dd00-db96-4388-91bd-25e6e7a6f8b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:20:09.448889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:09.450587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:09.451844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:09.451844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:09.451844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:20:09.451844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:20:09.451844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:20:09.451844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:20:09.451844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:20:09.451844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:20:09.451844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 05:20:09.451844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:09.451844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 627/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:09.456331 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a685a12-84db-4922-8f68-0fa4d9bc70e6 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:09.573240 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a685a12-84db-4922-8f68-0fa4d9bc70e6 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:09.613869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61b9a1-0752-4298-94c4-45a9a3d3fb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:09.614481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61b9a1-0752-4298-94c4-45a9a3d3fb8d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:09.614768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61b9a1-0752-4298-94c4-45a9a3d3fb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:09.615078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61b9a1-0752-4298-94c4-45a9a3d3fb8d None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:09.615769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61b9a1-0752-4298-94c4-45a9a3d3fb8d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:09.616115 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61b9a1-0752-4298-94c4-45a9a3d3fb8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:09.617623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61b9a1-0752-4298-94c4-45a9a3d3fb8d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:09.643068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 613/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 11 05:20:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:09.644234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67dd00-db96-4388-91bd-25e6e7a6f8b3 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:09.752896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67dd00-db96-4388-91bd-25e6e7a6f8b3 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:09.791084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f905a3b-8a00-431e-a069-8f7dc4758210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:09.791679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f905a3b-8a00-431e-a069-8f7dc4758210 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:09.792033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f905a3b-8a00-431e-a069-8f7dc4758210 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:09.792412 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f905a3b-8a00-431e-a069-8f7dc4758210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:09.861479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:09.868353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f905a3b-8a00-431e-a069-8f7dc4758210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:09.870094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f905a3b-8a00-431e-a069-8f7dc4758210 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:09.986446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 628/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:20:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:20:09.992535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61b9a1-0752-4298-94c4-45a9a3d3fb8d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:10.023136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:10.103063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61b9a1-0752-4298-94c4-45a9a3d3fb8d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:10.143947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d29ad0-2b36-4ca6-bc82-08894b1ac6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:10.144818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d29ad0-2b36-4ca6-bc82-08894b1ac6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:10.145099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d29ad0-2b36-4ca6-bc82-08894b1ac6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:10.145373 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d29ad0-2b36-4ca6-bc82-08894b1ac6c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:10.156710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d29ad0-2b36-4ca6-bc82-08894b1ac6c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:10.158343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d29ad0-2b36-4ca6-bc82-08894b1ac6c4 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-29d29ad0-2b36-4ca6-bc82-08894b1ac6c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:20:10.174781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:10.175980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:20:10.177145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:20:10.177145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:10.177145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:10.177145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 614/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:10.181857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f905a3b-8a00-431e-a069-8f7dc4758210 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:10.291945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:10.299581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f905a3b-8a00-431e-a069-8f7dc4758210 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:10.339408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48b9a4-f9e0-42a7-a89b-a8ec6e6fbc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:10.340771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48b9a4-f9e0-42a7-a89b-a8ec6e6fbc94 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:10.341155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48b9a4-f9e0-42a7-a89b-a8ec6e6fbc94 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:10.341475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48b9a4-f9e0-42a7-a89b-a8ec6e6fbc94 None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:10.342255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48b9a4-f9e0-42a7-a89b-a8ec6e6fbc94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:10.345543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48b9a4-f9e0-42a7-a89b-a8ec6e6fbc94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:10.345543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48b9a4-f9e0-42a7-a89b-a8ec6e6fbc94 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:10.361051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 629/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:20:10.366951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d29ad0-2b36-4ca6-bc82-08894b1ac6c4 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:10.466992 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:10.495635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d29ad0-2b36-4ca6-bc82-08894b1ac6c4 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:10.536282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0238b7-2743-4535-93f4-210f7cada4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:10.536856 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0238b7-2743-4535-93f4-210f7cada4ae None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:10.537138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0238b7-2743-4535-93f4-210f7cada4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:10.537411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0238b7-2743-4535-93f4-210f7cada4ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:10.545858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0238b7-2743-4535-93f4-210f7cada4ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:10.547249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0238b7-2743-4535-93f4-210f7cada4ae None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-de0238b7-2743-4535-93f4-210f7cada4ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:20:10.558591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:10.560025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:10.561426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:20:10.561426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:20:10.561426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:20:10.561426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:20:10.561426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:20:10.561426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:20:10.561426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:20:10.561426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:10.564618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 615/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:20:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:10.565793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48b9a4-f9e0-42a7-a89b-a8ec6e6fbc94 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:10.685032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48b9a4-f9e0-42a7-a89b-a8ec6e6fbc94 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:10.723163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492867f2-5613-462f-93a2-4e6080c897d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:10.723778 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492867f2-5613-462f-93a2-4e6080c897d5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:10.724084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492867f2-5613-462f-93a2-4e6080c897d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:10.724356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492867f2-5613-462f-93a2-4e6080c897d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:10.725053 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492867f2-5613-462f-93a2-4e6080c897d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:10.725363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492867f2-5613-462f-93a2-4e6080c897d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:10.726852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492867f2-5613-462f-93a2-4e6080c897d5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:10.739895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 630/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:20:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:10.746027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0238b7-2743-4535-93f4-210f7cada4ae None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:10.862109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0238b7-2743-4535-93f4-210f7cada4ae None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:10.897934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb494dc8-58be-49fe-843b-3fdde8ac2f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:10.898609 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb494dc8-58be-49fe-843b-3fdde8ac2f6a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:10.898898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb494dc8-58be-49fe-843b-3fdde8ac2f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:10.899179 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb494dc8-58be-49fe-843b-3fdde8ac2f6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:10.899951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb494dc8-58be-49fe-843b-3fdde8ac2f6a None admin] RBAC: Adding query filter params (role.id=7a06d87f01d940b38065f296fa63d0f7, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=4e985df8ad6c4bc7b3cd1328924e3ce2) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:10.900289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb494dc8-58be-49fe-843b-3fdde8ac2f6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:10.901838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb494dc8-58be-49fe-843b-3fdde8ac2f6a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:10.912810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 616/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:20:10 2022] GET /identity/v3/role_assignments?user.id=4e985df8ad6c4bc7b3cd1328924e3ce2&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=7a06d87f01d940b38065f296fa63d0f7 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:13.437754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9009ba-06f0-436d-865f-fecbea22f1ce None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:13.438330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9009ba-06f0-436d-865f-fecbea22f1ce None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:13.438330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9009ba-06f0-436d-865f-fecbea22f1ce None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:13.439645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 631/1247] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:20:13.445108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e1478-ed24-477a-8e8c-57e40617a455 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:13.445498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e1478-ed24-477a-8e8c-57e40617a455 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:13.445660 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e1478-ed24-477a-8e8c-57e40617a455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:13.453850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:13.468151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-704e1478-ed24-477a-8e8c-57e40617a455 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:20:13.469336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-704e1478-ed24-477a-8e8c-57e40617a455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:20:13.541727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 617/1248] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:20:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:20:13.830946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9009ba-06f0-436d-865f-fecbea22f1ce None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:13.850709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:13.946854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9009ba-06f0-436d-865f-fecbea22f1ce None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:13.984851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0a490-0f99-476d-93ba-01f3be09d664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:13.985515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0a490-0f99-476d-93ba-01f3be09d664 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:13.985781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0a490-0f99-476d-93ba-01f3be09d664 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:13.986049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0a490-0f99-476d-93ba-01f3be09d664 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:13.997942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0a490-0f99-476d-93ba-01f3be09d664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5db0a490-0f99-476d-93ba-01f3be09d664 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:20:14.003018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:14.004227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:14.005369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:14.005369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:14.005369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:14.005369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:14.005369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:20:14.005369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:20:14.005369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:20:14.005369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:20:14.005369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:20:14.005369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:20:14.005369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:20:14.005369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:14.005369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 632/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:20:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:14.011005 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e1478-ed24-477a-8e8c-57e40617a455 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:14.121948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e1478-ed24-477a-8e8c-57e40617a455 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:14.153801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:14.159461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a6c1c-d341-478d-a343-3c53e31dd615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:14.160128 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a6c1c-d341-478d-a343-3c53e31dd615 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:14.160414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a6c1c-d341-478d-a343-3c53e31dd615 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:14.160689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a6c1c-d341-478d-a343-3c53e31dd615 None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:14.161368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a6c1c-d341-478d-a343-3c53e31dd615 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:14.161675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a6c1c-d341-478d-a343-3c53e31dd615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:14.163082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a6c1c-d341-478d-a343-3c53e31dd615 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:14.172858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 618/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:20:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:14.178770 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0a490-0f99-476d-93ba-01f3be09d664 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:14.260871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:14.285240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0a490-0f99-476d-93ba-01f3be09d664 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:14.320897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd71512-ea75-42f8-a0a0-b51805ede7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:14.321544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd71512-ea75-42f8-a0a0-b51805ede7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:14.321818 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd71512-ea75-42f8-a0a0-b51805ede7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:14.322086 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd71512-ea75-42f8-a0a0-b51805ede7a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:14.334814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd71512-ea75-42f8-a0a0-b51805ede7a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:14.336708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd71512-ea75-42f8-a0a0-b51805ede7a4 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5dd71512-ea75-42f8-a0a0-b51805ede7a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:20:14.353305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:14.354677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:20:14.355806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:20:14.355806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:14.355806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:14.355806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 633/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:14.361433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a6c1c-d341-478d-a343-3c53e31dd615 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:14.475207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a6c1c-d341-478d-a343-3c53e31dd615 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:14.510976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2833b7-fafc-453e-8e90-b21bad699c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:14.511581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2833b7-fafc-453e-8e90-b21bad699c5b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:14.511885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2833b7-fafc-453e-8e90-b21bad699c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:14.512414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2833b7-fafc-453e-8e90-b21bad699c5b None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:14.513131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2833b7-fafc-453e-8e90-b21bad699c5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:14.513435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2833b7-fafc-453e-8e90-b21bad699c5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:14.514804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2833b7-fafc-453e-8e90-b21bad699c5b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:14.527182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 619/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:14.533205 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd71512-ea75-42f8-a0a0-b51805ede7a4 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:14.637944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd71512-ea75-42f8-a0a0-b51805ede7a4 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:14.663545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:14.672206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320b07b-fb60-49f4-8674-b85e9c57b5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:14.672833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320b07b-fb60-49f4-8674-b85e9c57b5ed None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:14.673111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320b07b-fb60-49f4-8674-b85e9c57b5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:14.673379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320b07b-fb60-49f4-8674-b85e9c57b5ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:14.680646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320b07b-fb60-49f4-8674-b85e9c57b5ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:14.682084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320b07b-fb60-49f4-8674-b85e9c57b5ed None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-d320b07b-fb60-49f4-8674-b85e9c57b5ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:20:14.693311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:14.698938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:14.700615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:14.700615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:14.700615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:20:14.700615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:20:14.700615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:20:14.700615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:20:14.700615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:20:14.700615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:20:14.700615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 05:20:14.700615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:14.700615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 634/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:14.701357 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2833b7-fafc-453e-8e90-b21bad699c5b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:14.755029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:14.799049 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2833b7-fafc-453e-8e90-b21bad699c5b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:14.832949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc6acc-e585-4795-9b4c-f78377fb9a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:14.833552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc6acc-e585-4795-9b4c-f78377fb9a2f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:14.833830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc6acc-e585-4795-9b4c-f78377fb9a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:14.834102 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc6acc-e585-4795-9b4c-f78377fb9a2f None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:14.834821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc6acc-e585-4795-9b4c-f78377fb9a2f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:14.835124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc6acc-e585-4795-9b4c-f78377fb9a2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:14.836599 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc6acc-e585-4795-9b4c-f78377fb9a2f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:14.853984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 620/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 11 05:20:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:20:14.860295 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320b07b-fb60-49f4-8674-b85e9c57b5ed None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:14.955724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320b07b-fb60-49f4-8674-b85e9c57b5ed None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:14.988075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b7d56-c270-4d14-960f-4528dbd963be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:14.988652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b7d56-c270-4d14-960f-4528dbd963be None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:14.988929 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b7d56-c270-4d14-960f-4528dbd963be None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:14.989196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b7d56-c270-4d14-960f-4528dbd963be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:15.041256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:15.053806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b7d56-c270-4d14-960f-4528dbd963be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:15.055236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b7d56-c270-4d14-960f-4528dbd963be None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:15.148344 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 635/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:20:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:20:15.154599 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc6acc-e585-4795-9b4c-f78377fb9a2f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:15.250752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc6acc-e585-4795-9b4c-f78377fb9a2f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:15.264141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:15.285563 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e396e71-5755-4773-8e51-c596d8a52c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:15.286184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e396e71-5755-4773-8e51-c596d8a52c28 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:15.286463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e396e71-5755-4773-8e51-c596d8a52c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:15.286736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e396e71-5755-4773-8e51-c596d8a52c28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:15.296559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e396e71-5755-4773-8e51-c596d8a52c28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:15.297949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e396e71-5755-4773-8e51-c596d8a52c28 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-2e396e71-5755-4773-8e51-c596d8a52c28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:20:15.312199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:15.313338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:20:15.314456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:20:15.314456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:15.314456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:15.314456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 621/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:15.318884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b7d56-c270-4d14-960f-4528dbd963be None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:15.409393 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:15.421070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b7d56-c270-4d14-960f-4528dbd963be None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:15.453436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49dd55-998d-4212-8c18-ca7510c30dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:15.454029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49dd55-998d-4212-8c18-ca7510c30dae None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:15.454308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49dd55-998d-4212-8c18-ca7510c30dae None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:15.454581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49dd55-998d-4212-8c18-ca7510c30dae None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:15.455208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49dd55-998d-4212-8c18-ca7510c30dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:15.455507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49dd55-998d-4212-8c18-ca7510c30dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:15.456882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49dd55-998d-4212-8c18-ca7510c30dae None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:15.468424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 636/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:15.474150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e396e71-5755-4773-8e51-c596d8a52c28 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:15.571518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e396e71-5755-4773-8e51-c596d8a52c28 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:15.604711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d721051-bec5-4bc5-ba92-abed3bff3c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:15.605316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d721051-bec5-4bc5-ba92-abed3bff3c1a None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:15.605624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d721051-bec5-4bc5-ba92-abed3bff3c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:15.605890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d721051-bec5-4bc5-ba92-abed3bff3c1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:15.612266 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d721051-bec5-4bc5-ba92-abed3bff3c1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:15.613697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d721051-bec5-4bc5-ba92-abed3bff3c1a None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-1d721051-bec5-4bc5-ba92-abed3bff3c1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:20:15.625174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:15.626225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:15.627345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:20:15.627345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:20:15.627345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:20:15.627345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:20:15.627345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:20:15.627345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:20:15.627345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:20:15.627345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:15.627345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 622/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:20:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:15.632445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49dd55-998d-4212-8c18-ca7510c30dae None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:15.728250 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49dd55-998d-4212-8c18-ca7510c30dae None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:15.760290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef730b-8d2d-4cca-9957-8bea42a4ec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:15.760886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef730b-8d2d-4cca-9957-8bea42a4ec06 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:15.761162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef730b-8d2d-4cca-9957-8bea42a4ec06 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:15.761427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef730b-8d2d-4cca-9957-8bea42a4ec06 None admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:15.762773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef730b-8d2d-4cca-9957-8bea42a4ec06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:15.763074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef730b-8d2d-4cca-9957-8bea42a4ec06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:15.764527 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef730b-8d2d-4cca-9957-8bea42a4ec06 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:15.775819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 637/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:20:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:15.782056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d721051-bec5-4bc5-ba92-abed3bff3c1a None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:15.807383 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:15.881419 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d721051-bec5-4bc5-ba92-abed3bff3c1a None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:15.914021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b542bc8-66ba-4e68-88d9-8e394edc6d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:15.914611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b542bc8-66ba-4e68-88d9-8e394edc6d42 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:15.914893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b542bc8-66ba-4e68-88d9-8e394edc6d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:15.915165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b542bc8-66ba-4e68-88d9-8e394edc6d42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:15.915878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b542bc8-66ba-4e68-88d9-8e394edc6d42 None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=4e985df8ad6c4bc7b3cd1328924e3ce2) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:15.916210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b542bc8-66ba-4e68-88d9-8e394edc6d42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:15.917645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b542bc8-66ba-4e68-88d9-8e394edc6d42 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:15.922959 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 623/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:20:15 2022] GET /identity/v3/role_assignments?user.id=4e985df8ad6c4bc7b3cd1328924e3ce2&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:18.063204 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622ae0-db86-464e-87df-32af125fba06 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:18.063705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622ae0-db86-464e-87df-32af125fba06 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:18.063705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622ae0-db86-464e-87df-32af125fba06 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:18.065100 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 638/1261] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:20:18.070176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30d13c-147c-4a05-8280-1d361e0ecad1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:18.070579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30d13c-147c-4a05-8280-1d361e0ecad1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:18.070717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30d13c-147c-4a05-8280-1d361e0ecad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:18.089628 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-da30d13c-147c-4a05-8280-1d361e0ecad1 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:20:18.091674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-da30d13c-147c-4a05-8280-1d361e0ecad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:20:18.156865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 624/1262] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:20:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:20:18.446262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622ae0-db86-464e-87df-32af125fba06 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:18.457758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:18.546690 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622ae0-db86-464e-87df-32af125fba06 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:18.579159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3547a-6f63-4c08-a103-d1bf0a4c0f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:18.579777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3547a-6f63-4c08-a103-d1bf0a4c0f35 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:18.580083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3547a-6f63-4c08-a103-d1bf0a4c0f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:18.580356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3547a-6f63-4c08-a103-d1bf0a4c0f35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:18.590669 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d3547a-6f63-4c08-a103-d1bf0a4c0f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-88d3547a-6f63-4c08-a103-d1bf0a4c0f35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:20:18.595589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:18.600048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:18.601184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:18.601184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:18.601184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:18.601184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:18.601184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:20:18.601184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:20:18.601184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:20:18.601184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:20:18.601184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:20:18.601184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:20:18.601184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:20:18.601184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:18.601184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 639/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:20:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:18.602780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30d13c-147c-4a05-8280-1d361e0ecad1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:18.620749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:18.702889 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30d13c-147c-4a05-8280-1d361e0ecad1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:18.735904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939bfc2-593a-4447-9f0b-9cd40da57a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:18.736527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939bfc2-593a-4447-9f0b-9cd40da57a0c None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:18.736810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939bfc2-593a-4447-9f0b-9cd40da57a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:18.737084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939bfc2-593a-4447-9f0b-9cd40da57a0c None admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:18.737744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939bfc2-593a-4447-9f0b-9cd40da57a0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:18.738048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939bfc2-593a-4447-9f0b-9cd40da57a0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:18.739412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939bfc2-593a-4447-9f0b-9cd40da57a0c None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:18.748103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 625/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:20:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:18.753757 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3547a-6f63-4c08-a103-d1bf0a4c0f35 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:18.818989 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:18.852281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3547a-6f63-4c08-a103-d1bf0a4c0f35 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:18.884487 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f1761-9525-45eb-acfe-27a35a3d669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:18.885133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f1761-9525-45eb-acfe-27a35a3d669e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:18.885413 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f1761-9525-45eb-acfe-27a35a3d669e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:18.885686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f1761-9525-45eb-acfe-27a35a3d669e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:18.896371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f1761-9525-45eb-acfe-27a35a3d669e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:18.897812 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f1761-9525-45eb-acfe-27a35a3d669e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-9b2f1761-9525-45eb-acfe-27a35a3d669e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:20:18.911378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:18.912535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:20:18.913530 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:20:18.913530 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:18.913530 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:18.913530 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 640/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:18.918501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939bfc2-593a-4447-9f0b-9cd40da57a0c None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:18.999159 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:19.018404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939bfc2-593a-4447-9f0b-9cd40da57a0c None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:19.052762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc633c-d4ca-4185-aafe-1a8caeb14d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:19.053355 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc633c-d4ca-4185-aafe-1a8caeb14d71 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:19.053636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc633c-d4ca-4185-aafe-1a8caeb14d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:19.053908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc633c-d4ca-4185-aafe-1a8caeb14d71 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:19.054544 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc633c-d4ca-4185-aafe-1a8caeb14d71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:19.054847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc633c-d4ca-4185-aafe-1a8caeb14d71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:19.056239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc633c-d4ca-4185-aafe-1a8caeb14d71 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:19.068379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 626/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:19.074193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f1761-9525-45eb-acfe-27a35a3d669e None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:19.170298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f1761-9525-45eb-acfe-27a35a3d669e None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:19.189162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:19.202736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041a5f4-d437-4454-aace-914daca90fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:19.203348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041a5f4-d437-4454-aace-914daca90fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:19.203624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041a5f4-d437-4454-aace-914daca90fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:19.203894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041a5f4-d437-4454-aace-914daca90fd5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:19.210955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041a5f4-d437-4454-aace-914daca90fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:19.212415 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041a5f4-d437-4454-aace-914daca90fd5 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-5041a5f4-d437-4454-aace-914daca90fd5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:20:19.223247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:19.224394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:19.225730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:19.225730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:19.225730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:20:19.225730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:20:19.225730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:20:19.225730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:20:19.225730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:20:19.225730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:20:19.225730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 05:20:19.225730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:19.225730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 641/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:19.230142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc633c-d4ca-4185-aafe-1a8caeb14d71 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:19.329147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc633c-d4ca-4185-aafe-1a8caeb14d71 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:19.365327 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0efd16-70ae-4992-b590-ce5804f4e08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:19.365928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0efd16-70ae-4992-b590-ce5804f4e08b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:19.366200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0efd16-70ae-4992-b590-ce5804f4e08b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:19.366481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0efd16-70ae-4992-b590-ce5804f4e08b None admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:19.367202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0efd16-70ae-4992-b590-ce5804f4e08b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:19.367501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0efd16-70ae-4992-b590-ce5804f4e08b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:19.368950 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0efd16-70ae-4992-b590-ce5804f4e08b None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:19.377954 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:19.387695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 627/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 11 05:20:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:19.393533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041a5f4-d437-4454-aace-914daca90fd5 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:19.490489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041a5f4-d437-4454-aace-914daca90fd5 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:19.522346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625074a4-0b70-4122-9725-6e0b579527e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:19.522961 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625074a4-0b70-4122-9725-6e0b579527e8 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:19.523238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625074a4-0b70-4122-9725-6e0b579527e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:19.523508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625074a4-0b70-4122-9725-6e0b579527e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:19.534254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625074a4-0b70-4122-9725-6e0b579527e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:19.535679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625074a4-0b70-4122-9725-6e0b579527e8 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-625074a4-0b70-4122-9725-6e0b579527e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:20:19.549875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:19.550995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:20:19.552147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:20:19.552147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:19.552147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:19.552147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 642/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:19.556623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0efd16-70ae-4992-b590-ce5804f4e08b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:19.654133 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0efd16-70ae-4992-b590-ce5804f4e08b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:19.687313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21403b-34d8-4513-b682-de2adde50455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:19.687904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21403b-34d8-4513-b682-de2adde50455 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:19.688209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21403b-34d8-4513-b682-de2adde50455 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:19.688482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21403b-34d8-4513-b682-de2adde50455 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:19.689117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21403b-34d8-4513-b682-de2adde50455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:19.689417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21403b-34d8-4513-b682-de2adde50455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:19.690777 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21403b-34d8-4513-b682-de2adde50455 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:19.701635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 628/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:19.707222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625074a4-0b70-4122-9725-6e0b579527e8 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:19.738525 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:19.805193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625074a4-0b70-4122-9725-6e0b579527e8 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:19.837825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c7ba2-2c90-4416-b58b-75af21172b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:19.838438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c7ba2-2c90-4416-b58b-75af21172b56 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:19.838717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c7ba2-2c90-4416-b58b-75af21172b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:19.838988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c7ba2-2c90-4416-b58b-75af21172b56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:19.846139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c7ba2-2c90-4416-b58b-75af21172b56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:19.847580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c7ba2-2c90-4416-b58b-75af21172b56 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-2e9c7ba2-2c90-4416-b58b-75af21172b56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:20:19.858576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:19.859638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:19.860599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:20:19.860599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:20:19.860599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:20:19.860599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:20:19.860599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:20:19.860599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:20:19.860599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:20:19.860599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:19.860599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 643/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:20:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:19.865380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21403b-34d8-4513-b682-de2adde50455 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:19.923296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:19.963927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21403b-34d8-4513-b682-de2adde50455 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:19.996428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab20ceb-eada-462f-8e1b-38b5eeaaf473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:19.997011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab20ceb-eada-462f-8e1b-38b5eeaaf473 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:19.997292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab20ceb-eada-462f-8e1b-38b5eeaaf473 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:19.997565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab20ceb-eada-462f-8e1b-38b5eeaaf473 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:19.998247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab20ceb-eada-462f-8e1b-38b5eeaaf473 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:19.998552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab20ceb-eada-462f-8e1b-38b5eeaaf473 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:19.999976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab20ceb-eada-462f-8e1b-38b5eeaaf473 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:20.012296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 629/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:20:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:20.018260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c7ba2-2c90-4416-b58b-75af21172b56 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:20.107088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:20.115764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c7ba2-2c90-4416-b58b-75af21172b56 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:20.148407 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd4eb2-a37e-4ac2-913d-abc693cac21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:20.149045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd4eb2-a37e-4ac2-913d-abc693cac21f None admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:20.149323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd4eb2-a37e-4ac2-913d-abc693cac21f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:20.149592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd4eb2-a37e-4ac2-913d-abc693cac21f None admin] PATH_INFO: `/v3/projects/092dd30ff8b541e2a45947504831998a/users/4e985df8ad6c4bc7b3cd1328924e3ce2/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:20.151451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd4eb2-a37e-4ac2-913d-abc693cac21f None admin] RBAC: Authorizing `identity:create_grant(project_id=092dd30ff8b541e2a45947504831998a, user_id=4e985df8ad6c4bc7b3cd1328924e3ce2, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:20.153435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd4eb2-a37e-4ac2-913d-abc693cac21f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:20.165385 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 644/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Sun Sep 11 05:20:20 2022] PUT /identity/v3/projects/092dd30ff8b541e2a45947504831998a/users/4e985df8ad6c4bc7b3cd1328924e3ce2/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:20:22.272325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b54c26-0e6d-4282-ab2d-bad1b418887e None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:22.272852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b54c26-0e6d-4282-ab2d-bad1b418887e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:22.272852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b54c26-0e6d-4282-ab2d-bad1b418887e None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:22.274201 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 630/1274] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:20:22.280062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb8a88-2e3d-4738-be01-01f194d7013b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:22.280442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb8a88-2e3d-4738-be01-01f194d7013b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:22.280606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb8a88-2e3d-4738-be01-01f194d7013b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:22.299468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4efb8a88-2e3d-4738-be01-01f194d7013b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:20:22.300704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4efb8a88-2e3d-4738-be01-01f194d7013b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:20:22.366442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 645/1275] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:20:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:20:22.640203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b54c26-0e6d-4282-ab2d-bad1b418887e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:22.740438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b54c26-0e6d-4282-ab2d-bad1b418887e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:22.757936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:22.774804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c06091-f2b8-4814-bf74-d37d179c705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:22.775427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c06091-f2b8-4814-bf74-d37d179c705b None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:22.775708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c06091-f2b8-4814-bf74-d37d179c705b None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:22.775979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c06091-f2b8-4814-bf74-d37d179c705b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:22.786223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c06091-f2b8-4814-bf74-d37d179c705b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c4c06091-f2b8-4814-bf74-d37d179c705b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 05:20:22.791075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:22.792186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:22.794084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:22.794084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:22.794084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:22.794084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:22.794084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 05:20:22.794084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 05:20:22.794084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 05:20:22.794084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 05:20:22.794084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 05:20:22.794084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 05:20:22.794084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 05:20:22.794084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:22.794084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 631/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Sun Sep 11 05:20:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:22.799128 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb8a88-2e3d-4738-be01-01f194d7013b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:22.876696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:22.895349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb8a88-2e3d-4738-be01-01f194d7013b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:22.927571 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc585ac2-84cb-4792-a995-b563a3f82934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:22.928193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc585ac2-84cb-4792-a995-b563a3f82934 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:22.928474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc585ac2-84cb-4792-a995-b563a3f82934 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:22.928745 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc585ac2-84cb-4792-a995-b563a3f82934 None admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:22.929401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc585ac2-84cb-4792-a995-b563a3f82934 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:22.929701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc585ac2-84cb-4792-a995-b563a3f82934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:22.931056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc585ac2-84cb-4792-a995-b563a3f82934 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:22.939414 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 646/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Sun Sep 11 05:20:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:22.945202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c06091-f2b8-4814-bf74-d37d179c705b None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:23.047284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c06091-f2b8-4814-bf74-d37d179c705b None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:23.080546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847d10d-93f6-43a5-b4df-349a239653bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:23.081159 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847d10d-93f6-43a5-b4df-349a239653bf None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:23.081440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847d10d-93f6-43a5-b4df-349a239653bf None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:23.081711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847d10d-93f6-43a5-b4df-349a239653bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:23.092040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847d10d-93f6-43a5-b4df-349a239653bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:23.093479 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847d10d-93f6-43a5-b4df-349a239653bf None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-7847d10d-93f6-43a5-b4df-349a239653bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:20:23.107689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:23.108923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:20:23.110325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:20:23.110325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:23.110325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:23.110325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 632/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:23.114576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc585ac2-84cb-4792-a995-b563a3f82934 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:23.212057 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc585ac2-84cb-4792-a995-b563a3f82934 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:23.243744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6b2f6-0d29-44d0-a32f-edf9ae0f4fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:23.244379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6b2f6-0d29-44d0-a32f-edf9ae0f4fda None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:23.244659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6b2f6-0d29-44d0-a32f-edf9ae0f4fda None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:23.244931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6b2f6-0d29-44d0-a32f-edf9ae0f4fda None admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:23.245567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6b2f6-0d29-44d0-a32f-edf9ae0f4fda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:23.246394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6b2f6-0d29-44d0-a32f-edf9ae0f4fda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:23.247748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6b2f6-0d29-44d0-a32f-edf9ae0f4fda None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:23.259368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 647/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:23.265120 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847d10d-93f6-43a5-b4df-349a239653bf None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:23.296145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:23.368991 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847d10d-93f6-43a5-b4df-349a239653bf None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:23.401841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32337ead-24b1-455f-8f4d-0fa635990855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:23.402989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337ead-24b1-455f-8f4d-0fa635990855 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:23.403259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337ead-24b1-455f-8f4d-0fa635990855 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:23.403524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32337ead-24b1-455f-8f4d-0fa635990855 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:23.410656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32337ead-24b1-455f-8f4d-0fa635990855 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:23.412133 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32337ead-24b1-455f-8f4d-0fa635990855 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-32337ead-24b1-455f-8f4d-0fa635990855 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 05:20:23.423498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:23.424661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:23.425627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:23.425627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:23.425627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 05:20:23.425627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 05:20:23.425627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 05:20:23.425627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 05:20:23.425627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 05:20:23.425627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 05:20:23.425627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 05:20:23.425627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:23.425627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 633/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:23.430397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6b2f6-0d29-44d0-a32f-edf9ae0f4fda None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:23.434126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:23.530320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6b2f6-0d29-44d0-a32f-edf9ae0f4fda None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:23.562339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee65519-aae2-44bb-b822-8f481280bb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:23.562942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee65519-aae2-44bb-b822-8f481280bb2f None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:23.563218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee65519-aae2-44bb-b822-8f481280bb2f None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:23.563485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee65519-aae2-44bb-b822-8f481280bb2f None admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:23.564231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee65519-aae2-44bb-b822-8f481280bb2f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:23.564534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee65519-aae2-44bb-b822-8f481280bb2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:23.565948 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee65519-aae2-44bb-b822-8f481280bb2f None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:23.582969 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 648/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Sun Sep 11 05:20:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:23.588812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32337ead-24b1-455f-8f4d-0fa635990855 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:23.679348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:23.689354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32337ead-24b1-455f-8f4d-0fa635990855 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:23.722640 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf3740-e8f9-4d99-9b8b-0f6087035059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:23.723225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf3740-e8f9-4d99-9b8b-0f6087035059 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:23.723503 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf3740-e8f9-4d99-9b8b-0f6087035059 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:23.723775 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf3740-e8f9-4d99-9b8b-0f6087035059 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:23.790217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf3740-e8f9-4d99-9b8b-0f6087035059 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:23.791601 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf3740-e8f9-4d99-9b8b-0f6087035059 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:23.880816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:23.891105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 634/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Sun Sep 11 05:20:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 05:20:23.896918 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee65519-aae2-44bb-b822-8f481280bb2f None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:23.956563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:23.995221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee65519-aae2-44bb-b822-8f481280bb2f None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:24.028129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c54ae-7353-4386-9a66-e002974cf48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:24.028740 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c54ae-7353-4386-9a66-e002974cf48d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:24.029019 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c54ae-7353-4386-9a66-e002974cf48d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:24.029287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c54ae-7353-4386-9a66-e002974cf48d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:24.038771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c54ae-7353-4386-9a66-e002974cf48d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:24.040293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c54ae-7353-4386-9a66-e002974cf48d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-733c54ae-7353-4386-9a66-e002974cf48d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 05:20:24.058548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:24.059774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 05:20:24.060919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 05:20:24.060919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 05:20:24.060919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:24.061109 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 649/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:24.067761 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf3740-e8f9-4d99-9b8b-0f6087035059 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:24.166682 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf3740-e8f9-4d99-9b8b-0f6087035059 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:24.200177 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f36f7e-505d-4455-84a2-b2a61491cc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:24.200765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f36f7e-505d-4455-84a2-b2a61491cc14 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:24.201043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f36f7e-505d-4455-84a2-b2a61491cc14 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:24.201313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f36f7e-505d-4455-84a2-b2a61491cc14 None admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:24.201945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f36f7e-505d-4455-84a2-b2a61491cc14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:24.202244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f36f7e-505d-4455-84a2-b2a61491cc14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:24.203613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f36f7e-505d-4455-84a2-b2a61491cc14 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:24.215196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 635/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Sun Sep 11 05:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:24.220818 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c54ae-7353-4386-9a66-e002974cf48d None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:24.317560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c54ae-7353-4386-9a66-e002974cf48d None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:24.330943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:24.353591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5c19d-11c2-4c55-a0a5-e1ac595e86ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:24.354215 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5c19d-11c2-4c55-a0a5-e1ac595e86ce None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:24.354492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5c19d-11c2-4c55-a0a5-e1ac595e86ce None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:24.354761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5c19d-11c2-4c55-a0a5-e1ac595e86ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:24.361447 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5c19d-11c2-4c55-a0a5-e1ac595e86ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:24.362863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5c19d-11c2-4c55-a0a5-e1ac595e86ce None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application [None req-a9a5c19d-11c2-4c55-a0a5-e1ac595e86ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 05:20:24.373741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:24.374779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:24.376103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 05:20:24.376103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 05:20:24.376103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 05:20:24.376103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 05:20:24.376103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 05:20:24.376103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 05:20:24.376103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 05:20:24.376103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: ERROR keystone.server.flask.application Sep 11 05:20:24.376103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 650/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Sun Sep 11 05:20:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:24.381018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f36f7e-505d-4455-84a2-b2a61491cc14 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:24.431113 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:24.481490 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f36f7e-505d-4455-84a2-b2a61491cc14 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:24.514435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8d2e5-1c87-4425-8aac-4b0526a70e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:24.515034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8d2e5-1c87-4425-8aac-4b0526a70e99 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:24.515313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8d2e5-1c87-4425-8aac-4b0526a70e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:24.515585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8d2e5-1c87-4425-8aac-4b0526a70e99 None admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:24.516301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8d2e5-1c87-4425-8aac-4b0526a70e99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:24.516606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8d2e5-1c87-4425-8aac-4b0526a70e99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:24.518057 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8d2e5-1c87-4425-8aac-4b0526a70e99 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:24.530221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 636/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Sun Sep 11 05:20:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:24.536334 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5c19d-11c2-4c55-a0a5-e1ac595e86ce None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:24.632678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5c19d-11c2-4c55-a0a5-e1ac595e86ce None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:24.665149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c7fdc8-2ff9-48be-8c98-665b16ae56a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:24.665739 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7fdc8-2ff9-48be-8c98-665b16ae56a3 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:24.666013 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7fdc8-2ff9-48be-8c98-665b16ae56a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:24.666279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7fdc8-2ff9-48be-8c98-665b16ae56a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:24.666987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c7fdc8-2ff9-48be-8c98-665b16ae56a3 None admin] RBAC: Adding query filter params (role.id=e1f556be9f8f4d1d889117e3a132132b, scope.project.id=092dd30ff8b541e2a45947504831998a, user.id=4e985df8ad6c4bc7b3cd1328924e3ce2) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:24.667289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c7fdc8-2ff9-48be-8c98-665b16ae56a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:24.669025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c7fdc8-2ff9-48be-8c98-665b16ae56a3 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:24.674513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 651/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Sun Sep 11 05:20:24 2022] GET /identity/v3/role_assignments?user.id=4e985df8ad6c4bc7b3cd1328924e3ce2&scope.project.id=092dd30ff8b541e2a45947504831998a&role.id=e1f556be9f8f4d1d889117e3a132132b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:26.844131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebd3b2-e18d-4916-bde3-e917ee43a390 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:26.844755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebd3b2-e18d-4916-bde3-e917ee43a390 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:26.844755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebd3b2-e18d-4916-bde3-e917ee43a390 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:26.846016 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 637/1288] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:20:26.851428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375dbf7-a8b6-435d-ab5d-45dc31b86180 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:26.851806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375dbf7-a8b6-435d-ab5d-45dc31b86180 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:26.851975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375dbf7-a8b6-435d-ab5d-45dc31b86180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:26.870372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f375dbf7-a8b6-435d-ab5d-45dc31b86180 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:20:26.871733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f375dbf7-a8b6-435d-ab5d-45dc31b86180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:20:26.900364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:26.937539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 652/1289] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:20:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:20:27.216878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebd3b2-e18d-4916-bde3-e917ee43a390 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:27.317031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebd3b2-e18d-4916-bde3-e917ee43a390 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:27.328273 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:27.352523 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1510b68-b3f2-47bb-9524-b9c168b73c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:27.353516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1510b68-b3f2-47bb-9524-b9c168b73c83 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:27.353771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1510b68-b3f2-47bb-9524-b9c168b73c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:27.354279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1510b68-b3f2-47bb-9524-b9c168b73c83 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:27.355600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1510b68-b3f2-47bb-9524-b9c168b73c83 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:27.357636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1510b68-b3f2-47bb-9524-b9c168b73c83 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-a1510b68-b3f2-47bb-9524-b9c168b73c83 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:27.366896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:27.368203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:27.369301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 638/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 05:20:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:27.375005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375dbf7-a8b6-435d-ab5d-45dc31b86180 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:27.479984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375dbf7-a8b6-435d-ab5d-45dc31b86180 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:27.513711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc080d1-91bc-48fc-a002-6fa34e16c632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:27.514315 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc080d1-91bc-48fc-a002-6fa34e16c632 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:27.514596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc080d1-91bc-48fc-a002-6fa34e16c632 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:27.514867 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc080d1-91bc-48fc-a002-6fa34e16c632 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:27.515602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc080d1-91bc-48fc-a002-6fa34e16c632 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:27.515910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc080d1-91bc-48fc-a002-6fa34e16c632 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:27.517301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc080d1-91bc-48fc-a002-6fa34e16c632 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:27.522040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 653/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Sun Sep 11 05:20:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:27.527883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1510b68-b3f2-47bb-9524-b9c168b73c83 None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:27.627033 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1510b68-b3f2-47bb-9524-b9c168b73c83 None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:27.661434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5993a3-0504-45f8-a613-83f917c7f34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:27.662022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5993a3-0504-45f8-a613-83f917c7f34d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:27.662300 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5993a3-0504-45f8-a613-83f917c7f34d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:27.662571 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5993a3-0504-45f8-a613-83f917c7f34d None admin] PATH_INFO: `/v3/services` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:27.663189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5993a3-0504-45f8-a613-83f917c7f34d None admin] RBAC: Adding query filter params (type=placement) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:27.663486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5993a3-0504-45f8-a613-83f917c7f34d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:27.664873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5993a3-0504-45f8-a613-83f917c7f34d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:27.670237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 639/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Sun Sep 11 05:20:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:29.798156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e2f17-884a-4184-8b0a-e631c8aac08c None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:29.798710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e2f17-884a-4184-8b0a-e631c8aac08c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:29.798710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e2f17-884a-4184-8b0a-e631c8aac08c None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:29.800043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 654/1293] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:20:29.805408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257c6f7-e598-4c1e-a4ee-4b9c36ddc899 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:29.805798 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257c6f7-e598-4c1e-a4ee-4b9c36ddc899 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:29.805956 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257c6f7-e598-4c1e-a4ee-4b9c36ddc899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:29.824373 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9257c6f7-e598-4c1e-a4ee-4b9c36ddc899 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:20:29.825551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9257c6f7-e598-4c1e-a4ee-4b9c36ddc899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:20:29.854469 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:29.892612 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 640/1294] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:20:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 05:20:30.158937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e2f17-884a-4184-8b0a-e631c8aac08c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:30.189577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:30.258862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e2f17-884a-4184-8b0a-e631c8aac08c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:30.290649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530ecdd-53a1-42dd-a35f-04a58dc70057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:30.291240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530ecdd-53a1-42dd-a35f-04a58dc70057 None admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:30.291518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530ecdd-53a1-42dd-a35f-04a58dc70057 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:30.291784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530ecdd-53a1-42dd-a35f-04a58dc70057 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:30.292962 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530ecdd-53a1-42dd-a35f-04a58dc70057 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:30.294320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530ecdd-53a1-42dd-a35f-04a58dc70057 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:30.307843 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 655/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Sun Sep 11 05:20:30 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:20:32.492757 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a5000-ab71-4465-92fb-e3159625c28c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:32.493295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a5000-ab71-4465-92fb-e3159625c28c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:32.493295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a5000-ab71-4465-92fb-e3159625c28c None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:32.494667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 641/1296] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:20:32.500105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e119415-537b-484d-96bf-8c6aed0e6d2d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:32.500524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e119415-537b-484d-96bf-8c6aed0e6d2d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:32.500654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e119415-537b-484d-96bf-8c6aed0e6d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:32.522882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9e119415-537b-484d-96bf-8c6aed0e6d2d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:20:32.524101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9e119415-537b-484d-96bf-8c6aed0e6d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:20:32.594138 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 656/1297] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:20:32 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:20:32.864276 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a5000-ab71-4465-92fb-e3159625c28c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:32.962707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a5000-ab71-4465-92fb-e3159625c28c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:32.996172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f7253-a77e-4b5e-8153-614e788afa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:32.996788 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f7253-a77e-4b5e-8153-614e788afa9f None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:32.997068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f7253-a77e-4b5e-8153-614e788afa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:32.997342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f7253-a77e-4b5e-8153-614e788afa9f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:32.997915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f7253-a77e-4b5e-8153-614e788afa9f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:32.999281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f7253-a77e-4b5e-8153-614e788afa9f None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-ba8f7253-a77e-4b5e-8153-614e788afa9f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:33.008032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:33.009189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:33.010249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 642/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 05:20:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:33.015495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e119415-537b-484d-96bf-8c6aed0e6d2d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:33.033652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:33.112149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e119415-537b-484d-96bf-8c6aed0e6d2d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:33.144136 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ac1a9-4f8a-40cf-9169-26b557821e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:33.144725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ac1a9-4f8a-40cf-9169-26b557821e38 None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:33.145001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ac1a9-4f8a-40cf-9169-26b557821e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:33.145270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ac1a9-4f8a-40cf-9169-26b557821e38 None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:33.145884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ac1a9-4f8a-40cf-9169-26b557821e38 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:33.146180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ac1a9-4f8a-40cf-9169-26b557821e38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:33.147519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ac1a9-4f8a-40cf-9169-26b557821e38 None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:33.152948 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 657/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Sun Sep 11 05:20:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:33.158873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f7253-a77e-4b5e-8153-614e788afa9f None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:33.190050 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:33.258123 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f7253-a77e-4b5e-8153-614e788afa9f None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:33.291178 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6976b2d-9ec4-401a-9ed8-929c06e78b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:33.291761 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6976b2d-9ec4-401a-9ed8-929c06e78b32 None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:33.292062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6976b2d-9ec4-401a-9ed8-929c06e78b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:33.292331 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6976b2d-9ec4-401a-9ed8-929c06e78b32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:33.293022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6976b2d-9ec4-401a-9ed8-929c06e78b32 None admin] RBAC: Adding query filter params (interface=public, service_id=3242cef437c04da5a09f21cdca344f1e, region_id=RegionOne) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:33.293318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6976b2d-9ec4-401a-9ed8-929c06e78b32 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:33.294667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6976b2d-9ec4-401a-9ed8-929c06e78b32 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:33.299444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 643/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Sun Sep 11 05:20:33 2022] GET /identity/v3/endpoints?service_id=3242cef437c04da5a09f21cdca344f1e&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:33.304978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ac1a9-4f8a-40cf-9169-26b557821e38 None admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:33.391354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:33.413806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ac1a9-4f8a-40cf-9169-26b557821e38 None admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:33.445914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca42c24-d4fa-449a-a061-0af1b652836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:33.446518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca42c24-d4fa-449a-a061-0af1b652836e None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:33.446795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca42c24-d4fa-449a-a061-0af1b652836e None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:33.447066 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca42c24-d4fa-449a-a061-0af1b652836e None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:33.447660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca42c24-d4fa-449a-a061-0af1b652836e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:33.449574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca42c24-d4fa-449a-a061-0af1b652836e None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:33.456121 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 658/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Sun Sep 11 05:20:33 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:20:35.569610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec51503-31ab-4c5b-8afa-4fcf19cace05 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:35.570138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec51503-31ab-4c5b-8afa-4fcf19cace05 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:35.570138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec51503-31ab-4c5b-8afa-4fcf19cace05 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:35.571490 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 644/1302] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:20:35.577348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95bf3a-1a8d-479f-8b81-9b2519655298 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:35.577739 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95bf3a-1a8d-479f-8b81-9b2519655298 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:35.577900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95bf3a-1a8d-479f-8b81-9b2519655298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:35.596339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5b95bf3a-1a8d-479f-8b81-9b2519655298 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:20:35.597521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5b95bf3a-1a8d-479f-8b81-9b2519655298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:20:35.661975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 659/1303] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:20:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 05:20:35.936084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec51503-31ab-4c5b-8afa-4fcf19cace05 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:36.029642 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:36.036733 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec51503-31ab-4c5b-8afa-4fcf19cace05 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:36.069504 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e45b39-fe5a-4946-b8de-0e144a64db7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:36.070118 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e45b39-fe5a-4946-b8de-0e144a64db7d None admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:36.070392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e45b39-fe5a-4946-b8de-0e144a64db7d None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:36.070657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e45b39-fe5a-4946-b8de-0e144a64db7d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:36.071223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e45b39-fe5a-4946-b8de-0e144a64db7d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:36.072609 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e45b39-fe5a-4946-b8de-0e144a64db7d None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application [None req-c5e45b39-fe5a-4946-b8de-0e144a64db7d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 05:20:36.081768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 05:20:36.083074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application with Lock( Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self._enter() Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application return self.creator() Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application created_value = creator( Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: ERROR keystone.server.flask.application Sep 11 05:20:36.084290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 645/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Sun Sep 11 05:20:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 05:20:36.088783 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95bf3a-1a8d-479f-8b81-9b2519655298 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:36.162126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:36.184348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95bf3a-1a8d-479f-8b81-9b2519655298 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:36.216947 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63464183-6bd8-4452-95b4-d93ed1faca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:36.217482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63464183-6bd8-4452-95b4-d93ed1faca7d None admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:36.217760 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63464183-6bd8-4452-95b4-d93ed1faca7d None admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:36.218030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63464183-6bd8-4452-95b4-d93ed1faca7d None admin] PATH_INFO: `/v3/services` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:36.218653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63464183-6bd8-4452-95b4-d93ed1faca7d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:20:36.218955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63464183-6bd8-4452-95b4-d93ed1faca7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:36.220343 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63464183-6bd8-4452-95b4-d93ed1faca7d None admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:36.225198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 660/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Sun Sep 11 05:20:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:20:36.231385 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e45b39-fe5a-4946-b8de-0e144a64db7d None admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:20:36.328604 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e45b39-fe5a-4946-b8de-0e144a64db7d None admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:20:36.361898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500bd6d-9de7-47b2-8627-7fbaa02218f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:20:36.362483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500bd6d-9de7-47b2-8627-7fbaa02218f6 None admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:20:36.362765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500bd6d-9de7-47b2-8627-7fbaa02218f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:20:36.363035 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500bd6d-9de7-47b2-8627-7fbaa02218f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:20:36.363681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500bd6d-9de7-47b2-8627-7fbaa02218f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:20:36.365056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500bd6d-9de7-47b2-8627-7fbaa02218f6 None admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:20:36.384157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 646/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:20:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:20:46.222367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=30 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:20:46.379579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:21:40.855538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596194f-68d1-4d65-ac7b-1007e551b5aa None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:40.855981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596194f-68d1-4d65-ac7b-1007e551b5aa None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:40.855981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596194f-68d1-4d65-ac7b-1007e551b5aa None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:40.862701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecec63d-1adf-4b8f-9568-9cb624b4fefb None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:40.863256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecec63d-1adf-4b8f-9568-9cb624b4fefb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:40.863256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecec63d-1adf-4b8f-9568-9cb624b4fefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:40.866094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 661/1307] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:21:40 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:21:40.961399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-aecec63d-1adf-4b8f-9568-9cb624b4fefb None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:21:40.963255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-aecec63d-1adf-4b8f-9568-9cb624b4fefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:41.070942 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 647/1308] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:21:41.371301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e01af-baf5-4f6a-9860-ac4f81fe98b8 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:41.371585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e01af-baf5-4f6a-9860-ac4f81fe98b8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:41.372111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e01af-baf5-4f6a-9860-ac4f81fe98b8 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:41.373552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 662/1309] 10.222.0.33 () {54 vars in 965 bytes} [Sun Sep 11 05:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:21:41.380643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc251201-18e9-4625-9897-08baa5d0e625 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:41.381256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc251201-18e9-4625-9897-08baa5d0e625 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:41.381603 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc251201-18e9-4625-9897-08baa5d0e625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:41.471080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-cc251201-18e9-4625-9897-08baa5d0e625 None None] MFA Rules not processed for user `0bf2a5de0e744505a1317ce9fa224643`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:21:41.472451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cc251201-18e9-4625-9897-08baa5d0e625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:41.496770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:21:41.560486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 648/1310] 10.222.0.33 () {58 vars in 1062 bytes} [Sun Sep 11 05:21:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:21:41.569477 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e01af-baf5-4f6a-9860-ac4f81fe98b8 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:21:41.571294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f26e01af-baf5-4f6a-9860-ac4f81fe98b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:41.688668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e01af-baf5-4f6a-9860-ac4f81fe98b8 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:21:41.769613 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db2d11-84a1-407e-9813-09d3f0aec7cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:21:41.770236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db2d11-84a1-407e-9813-09d3f0aec7cf service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:41.770602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db2d11-84a1-407e-9813-09d3f0aec7cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:41.770947 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db2d11-84a1-407e-9813-09d3f0aec7cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:41.772641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-18db2d11-84a1-407e-9813-09d3f0aec7cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:41.833231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:21:41.894031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db2d11-84a1-407e-9813-09d3f0aec7cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:21:41.895820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db2d11-84a1-407e-9813-09d3f0aec7cf service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:21:41.915518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 663/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:21:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:21:44.567830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d05a4-f440-4105-ace7-edb5bb50697d None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:44.568391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d05a4-f440-4105-ace7-edb5bb50697d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:44.568391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d05a4-f440-4105-ace7-edb5bb50697d None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:44.569735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 649/1312] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:21:44.575438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9e101-a17c-44ac-9df1-4c2aa7d09bc5 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:44.575851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9e101-a17c-44ac-9df1-4c2aa7d09bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:44.576016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9e101-a17c-44ac-9df1-4c2aa7d09bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:44.657785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-58d9e101-a17c-44ac-9df1-4c2aa7d09bc5 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:21:44.659018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-58d9e101-a17c-44ac-9df1-4c2aa7d09bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:44.754208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 664/1313] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:21:45.053700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b941b-7811-4146-883d-53418e36c740 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:45.053912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b941b-7811-4146-883d-53418e36c740 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:45.054186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b941b-7811-4146-883d-53418e36c740 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:45.055578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 650/1314] 10.222.0.33 () {54 vars in 965 bytes} [Sun Sep 11 05:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:21:45.062106 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea060add-9513-4a91-8598-17ba2347d3af None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:45.062464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea060add-9513-4a91-8598-17ba2347d3af None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:45.062807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea060add-9513-4a91-8598-17ba2347d3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:45.075159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:21:45.139112 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ea060add-9513-4a91-8598-17ba2347d3af None None] MFA Rules not processed for user `0bf2a5de0e744505a1317ce9fa224643`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:21:45.140363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ea060add-9513-4a91-8598-17ba2347d3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:45.214176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 665/1315] 10.222.0.33 () {58 vars in 1062 bytes} [Sun Sep 11 05:21:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:21:45.223159 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b941b-7811-4146-883d-53418e36c740 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:21:45.224721 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ef6b941b-7811-4146-883d-53418e36c740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:45.330879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b941b-7811-4146-883d-53418e36c740 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:21:45.406488 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:21:45.409516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095935ca-2e0a-4937-9496-ee690f516fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:21:45.410322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095935ca-2e0a-4937-9496-ee690f516fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:45.410654 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095935ca-2e0a-4937-9496-ee690f516fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:45.411131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095935ca-2e0a-4937-9496-ee690f516fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:45.413044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-095935ca-2e0a-4937-9496-ee690f516fa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:45.528423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095935ca-2e0a-4937-9496-ee690f516fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:21:45.529878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095935ca-2e0a-4937-9496-ee690f516fa6 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:21:45.548448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 651/1316] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:21:48.247585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2b47d-284d-498b-a802-26a3af414af5 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:48.248054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2b47d-284d-498b-a802-26a3af414af5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:48.248152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2b47d-284d-498b-a802-26a3af414af5 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:48.249506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 666/1317] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:21:48.254716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61a8b5-d704-4a10-8b82-b2489ea27c5d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:48.255098 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61a8b5-d704-4a10-8b82-b2489ea27c5d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:48.255284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61a8b5-d704-4a10-8b82-b2489ea27c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:48.322387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9d61a8b5-d704-4a10-8b82-b2489ea27c5d None None] MFA Rules not processed for user `f001865a40c84901a03e7897036c3892`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:21:48.323598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9d61a8b5-d704-4a10-8b82-b2489ea27c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:48.376652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:21:48.421936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 652/1318] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:21:48 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:21:48.709819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2b47d-284d-498b-a802-26a3af414af5 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:21:48.711340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5ed2b47d-284d-498b-a802-26a3af414af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:48.814760 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2b47d-284d-498b-a802-26a3af414af5 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:21:48.817587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:21:48.887968 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576ed34-b221-4636-849f-736b6f720b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:21:48.888592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576ed34-b221-4636-849f-736b6f720b86 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:48.888865 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576ed34-b221-4636-849f-736b6f720b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:48.889129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576ed34-b221-4636-849f-736b6f720b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:48.890722 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4576ed34-b221-4636-849f-736b6f720b86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:48.997809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576ed34-b221-4636-849f-736b6f720b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:21:48.999166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576ed34-b221-4636-849f-736b6f720b86 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:21:49.017583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 667/1319] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:21:48 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:21:51.701517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63f820-8c5a-4422-aa41-1e132c48a34a None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:51.702055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63f820-8c5a-4422-aa41-1e132c48a34a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:51.702055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63f820-8c5a-4422-aa41-1e132c48a34a None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:51.703438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 653/1320] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:21:51.708664 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1605f5-663c-4102-95be-3b0d949e00a7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:51.709076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1605f5-663c-4102-95be-3b0d949e00a7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:51.709231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1605f5-663c-4102-95be-3b0d949e00a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:51.750454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:21:51.776856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9a1605f5-663c-4102-95be-3b0d949e00a7 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:21:51.778064 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9a1605f5-663c-4102-95be-3b0d949e00a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:51.872589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 668/1321] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:21:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:21:52.161613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63f820-8c5a-4422-aa41-1e132c48a34a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:21:52.163176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fd63f820-8c5a-4422-aa41-1e132c48a34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:52.270101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63f820-8c5a-4422-aa41-1e132c48a34a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:21:52.343636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a118e-9096-4447-bcd7-ab3ca533456a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:21:52.344263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a118e-9096-4447-bcd7-ab3ca533456a service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:52.344541 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a118e-9096-4447-bcd7-ab3ca533456a service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:52.344808 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a118e-9096-4447-bcd7-ab3ca533456a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:52.346373 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-006a118e-9096-4447-bcd7-ab3ca533456a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:52.353650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:21:52.455359 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a118e-9096-4447-bcd7-ab3ca533456a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:21:52.456756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a118e-9096-4447-bcd7-ab3ca533456a service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:21:52.477113 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 654/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:21:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:21:55.261034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891855e-1717-49ae-acd2-8a4c84b1a179 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:55.261590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891855e-1717-49ae-acd2-8a4c84b1a179 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:55.261590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891855e-1717-49ae-acd2-8a4c84b1a179 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:55.262908 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 669/1323] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:21:55.268339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe685445-bc95-4584-b041-24b86e60259f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:55.268734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe685445-bc95-4584-b041-24b86e60259f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:55.268883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe685445-bc95-4584-b041-24b86e60259f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:55.340479 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-fe685445-bc95-4584-b041-24b86e60259f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:21:55.341701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fe685445-bc95-4584-b041-24b86e60259f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:55.406788 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:21:55.449892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 655/1324] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:21:55.731361 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891855e-1717-49ae-acd2-8a4c84b1a179 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:21:55.733007 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d891855e-1717-49ae-acd2-8a4c84b1a179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:55.840549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891855e-1717-49ae-acd2-8a4c84b1a179 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:21:55.854374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:21:55.917128 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78508-ca4a-4b0e-8593-a8533f897e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:21:55.917744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78508-ca4a-4b0e-8593-a8533f897e10 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:55.918024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78508-ca4a-4b0e-8593-a8533f897e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:55.918297 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78508-ca4a-4b0e-8593-a8533f897e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:55.919922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ceb78508-ca4a-4b0e-8593-a8533f897e10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:56.030429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78508-ca4a-4b0e-8593-a8533f897e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:21:56.031756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78508-ca4a-4b0e-8593-a8533f897e10 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:21:56.050909 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 670/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:21:58.271308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281935fc-6f21-4e1f-8004-5b13eccd4cfe None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:58.271838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281935fc-6f21-4e1f-8004-5b13eccd4cfe None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:58.271838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281935fc-6f21-4e1f-8004-5b13eccd4cfe None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:58.273242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 656/1326] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:21:58.278686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da39f3e-10e1-4895-b40a-c8dd1211362c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:58.279121 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da39f3e-10e1-4895-b40a-c8dd1211362c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:58.279216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da39f3e-10e1-4895-b40a-c8dd1211362c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:58.327465 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:21:58.346329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6da39f3e-10e1-4895-b40a-c8dd1211362c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:21:58.347547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6da39f3e-10e1-4895-b40a-c8dd1211362c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:58.454476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 671/1327] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:21:58.738008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281935fc-6f21-4e1f-8004-5b13eccd4cfe None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:21:58.739650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-281935fc-6f21-4e1f-8004-5b13eccd4cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:58.848297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281935fc-6f21-4e1f-8004-5b13eccd4cfe None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:21:58.923787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadc312-6aa5-4552-9194-892638fbc491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:21:58.924417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadc312-6aa5-4552-9194-892638fbc491 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:21:58.924693 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadc312-6aa5-4552-9194-892638fbc491 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:21:58.924961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadc312-6aa5-4552-9194-892638fbc491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:21:58.926538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ceadc312-6aa5-4552-9194-892638fbc491 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:21:58.944997 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:21:59.040667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadc312-6aa5-4552-9194-892638fbc491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:21:59.042160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadc312-6aa5-4552-9194-892638fbc491 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:21:59.062306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 657/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:01.287168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbaad3-629c-482b-bfcb-cb066cf09e35 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:01.287653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbaad3-629c-482b-bfcb-cb066cf09e35 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:01.287653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbaad3-629c-482b-bfcb-cb066cf09e35 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:01.289158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 672/1329] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:01.294828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223a46c-f67a-4f0c-b4f7-e5fc02d81405 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:01.295225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223a46c-f67a-4f0c-b4f7-e5fc02d81405 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:01.295374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223a46c-f67a-4f0c-b4f7-e5fc02d81405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:01.363818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e223a46c-f67a-4f0c-b4f7-e5fc02d81405 None None] MFA Rules not processed for user `f001865a40c84901a03e7897036c3892`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:01.365061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e223a46c-f67a-4f0c-b4f7-e5fc02d81405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:01.431036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:01.462920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 658/1330] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:01 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:01.745857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbaad3-629c-482b-bfcb-cb066cf09e35 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:01.747685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-07cbaad3-629c-482b-bfcb-cb066cf09e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:01.856036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbaad3-629c-482b-bfcb-cb066cf09e35 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:01.877670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:01.935299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55af3fa-8bef-4d30-a99a-44212ed3f1db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:01.935828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55af3fa-8bef-4d30-a99a-44212ed3f1db service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:01.936138 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55af3fa-8bef-4d30-a99a-44212ed3f1db service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:01.936411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55af3fa-8bef-4d30-a99a-44212ed3f1db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:01.937990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d55af3fa-8bef-4d30-a99a-44212ed3f1db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:02.048369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55af3fa-8bef-4d30-a99a-44212ed3f1db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:02.049733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55af3fa-8bef-4d30-a99a-44212ed3f1db service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:02.068402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 673/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:01 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:04.470205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28266522-dc8b-43b2-a42a-d5dc1bb8133a None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:04.470693 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28266522-dc8b-43b2-a42a-d5dc1bb8133a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:04.470804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28266522-dc8b-43b2-a42a-d5dc1bb8133a None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:04.472120 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 659/1332] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:04.477526 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f27078-f265-4641-b628-47c6b5510d19 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:04.477934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f27078-f265-4641-b628-47c6b5510d19 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:04.478045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f27078-f265-4641-b628-47c6b5510d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:04.540516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:04.543541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-79f27078-f265-4641-b628-47c6b5510d19 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:04.545094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-79f27078-f265-4641-b628-47c6b5510d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:04.640134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 674/1333] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:04.921346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28266522-dc8b-43b2-a42a-d5dc1bb8133a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:04.922878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-28266522-dc8b-43b2-a42a-d5dc1bb8133a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:05.026538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28266522-dc8b-43b2-a42a-d5dc1bb8133a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:05.099114 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394b66f-201d-4043-b220-5b74306844da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:05.099712 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394b66f-201d-4043-b220-5b74306844da service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:05.100018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394b66f-201d-4043-b220-5b74306844da service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:05.100297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394b66f-201d-4043-b220-5b74306844da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:05.101877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a394b66f-201d-4043-b220-5b74306844da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:05.130012 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:05.212847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394b66f-201d-4043-b220-5b74306844da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:05.214280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394b66f-201d-4043-b220-5b74306844da service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:05.232610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 660/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:07.437476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754e1e7-4b64-49f9-a45c-b617ccbcb8b1 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:07.438018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754e1e7-4b64-49f9-a45c-b617ccbcb8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:07.438018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754e1e7-4b64-49f9-a45c-b617ccbcb8b1 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:07.439354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 675/1335] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:07.444811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7fb99-cd56-415d-ad04-a42dd5c2cc44 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:07.445243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7fb99-cd56-415d-ad04-a42dd5c2cc44 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:07.445378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7fb99-cd56-415d-ad04-a42dd5c2cc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:07.512911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a6d7fb99-cd56-415d-ad04-a42dd5c2cc44 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:07.514111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a6d7fb99-cd56-415d-ad04-a42dd5c2cc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:07.588108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:07.610650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 661/1336] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:07.900389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754e1e7-4b64-49f9-a45c-b617ccbcb8b1 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:07.901935 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6754e1e7-4b64-49f9-a45c-b617ccbcb8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:08.005451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754e1e7-4b64-49f9-a45c-b617ccbcb8b1 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:08.035082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:08.081331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4451c-00f3-4654-9b23-e15fc4efeb1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:08.081940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4451c-00f3-4654-9b23-e15fc4efeb1a service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:08.082218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4451c-00f3-4654-9b23-e15fc4efeb1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:08.082487 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4451c-00f3-4654-9b23-e15fc4efeb1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:08.084088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-aea4451c-00f3-4654-9b23-e15fc4efeb1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:08.191544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4451c-00f3-4654-9b23-e15fc4efeb1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:08.192999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4451c-00f3-4654-9b23-e15fc4efeb1a service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:08.211299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 676/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:10.971780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dd4ef-251f-4964-879a-1a4d7b1b2025 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:10.972325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dd4ef-251f-4964-879a-1a4d7b1b2025 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:10.972434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dd4ef-251f-4964-879a-1a4d7b1b2025 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:10.973713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 662/1338] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:10.979552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b863-293b-47f4-9ea9-d6c25cc56941 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:10.979958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b863-293b-47f4-9ea9-d6c25cc56941 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:10.980141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b863-293b-47f4-9ea9-d6c25cc56941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:11.044868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9715b863-293b-47f4-9ea9-d6c25cc56941 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:11.046092 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9715b863-293b-47f4-9ea9-d6c25cc56941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:11.054346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:11.142015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 677/1339] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:11.430501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dd4ef-251f-4964-879a-1a4d7b1b2025 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:11.432072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e88dd4ef-251f-4964-879a-1a4d7b1b2025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:11.535088 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dd4ef-251f-4964-879a-1a4d7b1b2025 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:11.607540 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccade3-e85c-4a41-9a2c-68be01ad693f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:11.608169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccade3-e85c-4a41-9a2c-68be01ad693f service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:11.608453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccade3-e85c-4a41-9a2c-68be01ad693f service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:11.608723 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccade3-e85c-4a41-9a2c-68be01ad693f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:11.610296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d8ccade3-e85c-4a41-9a2c-68be01ad693f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:11.645775 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:11.719411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccade3-e85c-4a41-9a2c-68be01ad693f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:11.720794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccade3-e85c-4a41-9a2c-68be01ad693f service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:11.739274 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 663/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:15.937658 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6325343-07b2-4412-8771-125b24000b5b None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:15.938184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6325343-07b2-4412-8771-125b24000b5b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:15.938184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6325343-07b2-4412-8771-125b24000b5b None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:15.939530 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 678/1341] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:15.944742 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dca01-b4cd-45af-87af-6cc290713b93 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:15.945178 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dca01-b4cd-45af-87af-6cc290713b93 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:15.945280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dca01-b4cd-45af-87af-6cc290713b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:16.012621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e75dca01-b4cd-45af-87af-6cc290713b93 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:16.013733 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e75dca01-b4cd-45af-87af-6cc290713b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:16.100700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:16.112455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 664/1342] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:16.396985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6325343-07b2-4412-8771-125b24000b5b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:16.398520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c6325343-07b2-4412-8771-125b24000b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:16.503238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6325343-07b2-4412-8771-125b24000b5b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:16.543814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:16.580749 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8857-f9a8-4a2f-890f-833799688cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:16.581356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8857-f9a8-4a2f-890f-833799688cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:16.581659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8857-f9a8-4a2f-890f-833799688cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:16.581926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8857-f9a8-4a2f-890f-833799688cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:16.583500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5ccd8857-f9a8-4a2f-890f-833799688cf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:16.692763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd8857-f9a8-4a2f-890f-833799688cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:16.694124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd8857-f9a8-4a2f-890f-833799688cf0 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:16.712870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 679/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:19.471162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa1d61-ba11-4682-8aad-538b24cbfb14 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:19.471700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa1d61-ba11-4682-8aad-538b24cbfb14 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:19.471700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa1d61-ba11-4682-8aad-538b24cbfb14 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:19.472989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 665/1344] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:19.479304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5f1b3-de05-4229-a73c-6bad20146cbc None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:19.479747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5f1b3-de05-4229-a73c-6bad20146cbc None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:19.479930 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5f1b3-de05-4229-a73c-6bad20146cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:19.546919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-85a5f1b3-de05-4229-a73c-6bad20146cbc None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:19.548184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-85a5f1b3-de05-4229-a73c-6bad20146cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:19.565868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:19.648416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 680/1345] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:19.934837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa1d61-ba11-4682-8aad-538b24cbfb14 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:19.936497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-59aa1d61-ba11-4682-8aad-538b24cbfb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:20.044316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa1d61-ba11-4682-8aad-538b24cbfb14 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:20.123947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774d895-34ef-461e-9baa-40e1c0fa64be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:20.124573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774d895-34ef-461e-9baa-40e1c0fa64be service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:20.124849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774d895-34ef-461e-9baa-40e1c0fa64be service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:20.125116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774d895-34ef-461e-9baa-40e1c0fa64be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:20.126670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3774d895-34ef-461e-9baa-40e1c0fa64be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:20.170264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:20.238365 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774d895-34ef-461e-9baa-40e1c0fa64be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:20.239855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774d895-34ef-461e-9baa-40e1c0fa64be service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:20.260842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 666/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:24.378354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99748250-56cf-443c-9119-7876789b81e7 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:24.378900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99748250-56cf-443c-9119-7876789b81e7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:24.378900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99748250-56cf-443c-9119-7876789b81e7 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:24.387011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628104bd-0c8f-4173-bdd5-559b0aebe7ec None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:24.387493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628104bd-0c8f-4173-bdd5-559b0aebe7ec None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:24.387574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628104bd-0c8f-4173-bdd5-559b0aebe7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:24.395896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 681/1347] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:24 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:24.458096 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-628104bd-0c8f-4173-bdd5-559b0aebe7ec None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:24.459322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-628104bd-0c8f-4173-bdd5-559b0aebe7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:24.567818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 667/1348] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:24.568341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:24.875159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99748250-56cf-443c-9119-7876789b81e7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:24.876757 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-99748250-56cf-443c-9119-7876789b81e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:24.979469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99748250-56cf-443c-9119-7876789b81e7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:25.026917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:25.052854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4113b0c-547c-4c54-8223-06d331139860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:25.053379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4113b0c-547c-4c54-8223-06d331139860 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:25.053655 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4113b0c-547c-4c54-8223-06d331139860 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:25.053921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4113b0c-547c-4c54-8223-06d331139860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:25.056424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d4113b0c-547c-4c54-8223-06d331139860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:25.163192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4113b0c-547c-4c54-8223-06d331139860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:25.165452 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4113b0c-547c-4c54-8223-06d331139860 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:25.184146 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 682/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:30.116572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d114b-31a7-408f-b1c6-3b15066c79e8 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:30.117118 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d114b-31a7-408f-b1c6-3b15066c79e8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:30.117692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d114b-31a7-408f-b1c6-3b15066c79e8 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:30.119312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 668/1350] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 05:22:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:30.124461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aafef1-86b6-42e9-aaef-71f0eb55ab67 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:30.124882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aafef1-86b6-42e9-aaef-71f0eb55ab67 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:30.125015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aafef1-86b6-42e9-aaef-71f0eb55ab67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:30.238477 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-29aafef1-86b6-42e9-aaef-71f0eb55ab67 None None] MFA Rules not processed for user `4e985df8ad6c4bc7b3cd1328924e3ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:30.239697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-29aafef1-86b6-42e9-aaef-71f0eb55ab67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:30.297252 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:30.401574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 683/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 05:22:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:22:34.622043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffc982-9cd7-4ac1-b68a-65d6fccb6d1c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:34.622580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffc982-9cd7-4ac1-b68a-65d6fccb6d1c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:34.622699 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffc982-9cd7-4ac1-b68a-65d6fccb6d1c None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:34.624191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 669/1352] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 05:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:34.629398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e810dc5-251b-472a-9918-fc047f82ba52 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:34.629831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e810dc5-251b-472a-9918-fc047f82ba52 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:34.630173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e810dc5-251b-472a-9918-fc047f82ba52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:34.714786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7e810dc5-251b-472a-9918-fc047f82ba52 None None] MFA Rules not processed for user `4e985df8ad6c4bc7b3cd1328924e3ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:34.716028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7e810dc5-251b-472a-9918-fc047f82ba52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:34.795795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:34.836924 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 684/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 05:22:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:36.199613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88df95-1062-4184-86bb-d64d19f3e36b None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:36.200184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88df95-1062-4184-86bb-d64d19f3e36b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:36.200184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88df95-1062-4184-86bb-d64d19f3e36b None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:36.201523 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 670/1354] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 05:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:36.206425 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349e6d2-62b3-468f-8382-e00fc5dc675f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:36.206875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349e6d2-62b3-468f-8382-e00fc5dc675f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:36.207151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349e6d2-62b3-468f-8382-e00fc5dc675f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:36.286134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2349e6d2-62b3-468f-8382-e00fc5dc675f None None] MFA Rules not processed for user `4e985df8ad6c4bc7b3cd1328924e3ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:36.287328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2349e6d2-62b3-468f-8382-e00fc5dc675f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:36.397807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 685/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 05:22:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:40.160985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcbcb0-30d1-464b-8d6a-40fbf6fcc0e0 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:40.161532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcbcb0-30d1-464b-8d6a-40fbf6fcc0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:40.161532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcbcb0-30d1-464b-8d6a-40fbf6fcc0e0 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:40.162786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 671/1356] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:22:40.169254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241c3af-d77d-42f1-b112-cf9741f090e6 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:40.169765 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241c3af-d77d-42f1-b112-cf9741f090e6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:40.169857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241c3af-d77d-42f1-b112-cf9741f090e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:40.208019 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:40.242868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a241c3af-d77d-42f1-b112-cf9741f090e6 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:40.244486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a241c3af-d77d-42f1-b112-cf9741f090e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:40.356337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 686/1357] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:40.670389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112181f5-0933-4274-ab57-740c2f11cf7c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:40.670673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112181f5-0933-4274-ab57-740c2f11cf7c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:40.670949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112181f5-0933-4274-ab57-740c2f11cf7c None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:40.672561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 672/1358] 10.222.0.33 () {54 vars in 962 bytes} [Sun Sep 11 05:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:40.683101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ab19c-52d6-47d8-b379-c2c607ed0193 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:40.683494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ab19c-52d6-47d8-b379-c2c607ed0193 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:40.683868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ab19c-52d6-47d8-b379-c2c607ed0193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:40.765159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-873ab19c-52d6-47d8-b379-c2c607ed0193 None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:40.766457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-873ab19c-52d6-47d8-b379-c2c607ed0193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:40.832207 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:40.921001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 687/1359] 10.222.0.33 () {58 vars in 1059 bytes} [Sun Sep 11 05:22:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:40.928791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112181f5-0933-4274-ab57-740c2f11cf7c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:40.930816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-112181f5-0933-4274-ab57-740c2f11cf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:41.090699 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112181f5-0933-4274-ab57-740c2f11cf7c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:41.167777 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:41.210235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757380b-d821-4385-8073-6ea04fe92b09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:41.211471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757380b-d821-4385-8073-6ea04fe92b09 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:41.211827 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757380b-d821-4385-8073-6ea04fe92b09 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:41.212197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757380b-d821-4385-8073-6ea04fe92b09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:41.213897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4757380b-d821-4385-8073-6ea04fe92b09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:41.322263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4757380b-d821-4385-8073-6ea04fe92b09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:41.323984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4757380b-d821-4385-8073-6ea04fe92b09 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:41.344675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 673/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:43.745660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d813a-fad9-425b-877a-6595747f7a10 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:43.745660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d813a-fad9-425b-877a-6595747f7a10 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:43.746434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d813a-fad9-425b-877a-6595747f7a10 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:43.752850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ae265-d8bd-4031-8027-3c92fc86b10c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:43.752850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ae265-d8bd-4031-8027-3c92fc86b10c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:43.753556 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ae265-d8bd-4031-8027-3c92fc86b10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:43.758594 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 688/1361] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:43 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:43.824525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-501ae265-d8bd-4031-8027-3c92fc86b10c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:43.825807 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-501ae265-d8bd-4031-8027-3c92fc86b10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:43.868692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:43.932870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 674/1362] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:22:44.227269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d813a-fad9-425b-877a-6595747f7a10 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:44.228845 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-104d813a-fad9-425b-877a-6595747f7a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:44.408512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d813a-fad9-425b-877a-6595747f7a10 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:44.425478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:44.564423 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc274a-72e4-470d-b3af-766a9ea7a44b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:44.565034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc274a-72e4-470d-b3af-766a9ea7a44b service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:44.565310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc274a-72e4-470d-b3af-766a9ea7a44b service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:44.565606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc274a-72e4-470d-b3af-766a9ea7a44b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:44.567187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-51fc274a-72e4-470d-b3af-766a9ea7a44b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:44.669885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc274a-72e4-470d-b3af-766a9ea7a44b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:44.671432 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc274a-72e4-470d-b3af-766a9ea7a44b service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:44.690250 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 689/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:47.311980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b00071-c664-4914-a395-8c669738501c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:47.312577 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b00071-c664-4914-a395-8c669738501c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:47.312826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b00071-c664-4914-a395-8c669738501c None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:47.314393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 675/1364] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:47.319536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a3827-f1c1-4404-a98a-ec1be51e65c0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:47.319961 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a3827-f1c1-4404-a98a-ec1be51e65c0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:47.320159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a3827-f1c1-4404-a98a-ec1be51e65c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:47.336263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:47.392596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-039a3827-f1c1-4404-a98a-ec1be51e65c0 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:47.394477 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-039a3827-f1c1-4404-a98a-ec1be51e65c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:47.506075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 690/1365] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:47.798211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86d082-966c-4f13-836a-190fd5e8fd8c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:47.798503 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86d082-966c-4f13-836a-190fd5e8fd8c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:47.798781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86d082-966c-4f13-836a-190fd5e8fd8c None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:47.800429 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 676/1366] 10.222.0.33 () {54 vars in 962 bytes} [Sun Sep 11 05:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:47.808748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084b369-884f-48d0-b3ec-5055948d66ba None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:47.809030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084b369-884f-48d0-b3ec-5055948d66ba None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:47.809299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084b369-884f-48d0-b3ec-5055948d66ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:47.889660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f084b369-884f-48d0-b3ec-5055948d66ba None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:47.890864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f084b369-884f-48d0-b3ec-5055948d66ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:47.928752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:48.023287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 691/1367] 10.222.0.33 () {58 vars in 1059 bytes} [Sun Sep 11 05:22:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:48.031393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86d082-966c-4f13-836a-190fd5e8fd8c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:48.033178 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2b86d082-966c-4f13-836a-190fd5e8fd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:48.193190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86d082-966c-4f13-836a-190fd5e8fd8c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:48.215072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:48.316724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ce98d-2fff-4621-a0dc-67028976010d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:48.317468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ce98d-2fff-4621-a0dc-67028976010d service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:48.317742 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ce98d-2fff-4621-a0dc-67028976010d service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:48.318011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ce98d-2fff-4621-a0dc-67028976010d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:48.320416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fb5ce98d-2fff-4621-a0dc-67028976010d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:48.423412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ce98d-2fff-4621-a0dc-67028976010d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:48.425067 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ce98d-2fff-4621-a0dc-67028976010d service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:48.445267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 677/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:48.445462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c415af-91c6-4b69-9da8-c4394050354c None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:48.445894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c415af-91c6-4b69-9da8-c4394050354c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:48.445894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c415af-91c6-4b69-9da8-c4394050354c None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:48.447229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 692/1369] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:22:48.452563 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf59504-ec47-4bca-b41c-be7171200676 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:48.452854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf59504-ec47-4bca-b41c-be7171200676 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:48.453132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf59504-ec47-4bca-b41c-be7171200676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:48.487595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:48.531505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7bf59504-ec47-4bca-b41c-be7171200676 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:48.533348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7bf59504-ec47-4bca-b41c-be7171200676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:48.660964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 678/1370] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:48.962279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960d76a-aa8a-4dea-ac1c-491d567db4c2 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:48.962564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960d76a-aa8a-4dea-ac1c-491d567db4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:48.962838 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960d76a-aa8a-4dea-ac1c-491d567db4c2 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:48.964471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 693/1371] 10.222.0.33 () {54 vars in 964 bytes} [Sun Sep 11 05:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:48.969765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20d91a-cbb4-4fa2-ad7e-d3e8df095091 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:48.970053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20d91a-cbb4-4fa2-ad7e-d3e8df095091 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:48.970328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20d91a-cbb4-4fa2-ad7e-d3e8df095091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:49.050488 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5a20d91a-cbb4-4fa2-ad7e-d3e8df095091 None None] MFA Rules not processed for user `8ef93155a40f4c69b320ca962d4e8868`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:49.051712 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5a20d91a-cbb4-4fa2-ad7e-d3e8df095091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:49.102587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:49.138179 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 679/1372] 10.222.0.33 () {58 vars in 1061 bytes} [Sun Sep 11 05:22:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:49.145873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960d76a-aa8a-4dea-ac1c-491d567db4c2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:49.147371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a960d76a-aa8a-4dea-ac1c-491d567db4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:49.258645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960d76a-aa8a-4dea-ac1c-491d567db4c2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:49.302969 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:49.340434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe51cee-b58f-44dd-bbb6-be8fbd307b08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:49.341050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe51cee-b58f-44dd-bbb6-be8fbd307b08 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:49.341328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe51cee-b58f-44dd-bbb6-be8fbd307b08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:49.341599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe51cee-b58f-44dd-bbb6-be8fbd307b08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:49.343204 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ffe51cee-b58f-44dd-bbb6-be8fbd307b08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:49.494290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe51cee-b58f-44dd-bbb6-be8fbd307b08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:49.495940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe51cee-b58f-44dd-bbb6-be8fbd307b08 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:49.528403 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 694/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:50.707027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae78dfb-a055-40d1-a60c-3ccdfcb405dc None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:50.707577 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae78dfb-a055-40d1-a60c-3ccdfcb405dc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:50.707577 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae78dfb-a055-40d1-a60c-3ccdfcb405dc None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:50.708943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 680/1374] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:50.714798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2efaa5-ed40-48e2-bd86-5aa99d6d2781 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:50.715245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2efaa5-ed40-48e2-bd86-5aa99d6d2781 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:50.715422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2efaa5-ed40-48e2-bd86-5aa99d6d2781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:50.784106 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9d2efaa5-ed40-48e2-bd86-5aa99d6d2781 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:50.785390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9d2efaa5-ed40-48e2-bd86-5aa99d6d2781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:50.812995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:50.893705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 695/1375] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:51.200547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae78dfb-a055-40d1-a60c-3ccdfcb405dc None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:51.202065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cae78dfb-a055-40d1-a60c-3ccdfcb405dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:51.360694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae78dfb-a055-40d1-a60c-3ccdfcb405dc None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:51.423794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:51.494820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9edba-777f-4db3-83d8-e3005a52a620 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:51.495434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9edba-777f-4db3-83d8-e3005a52a620 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:51.495710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9edba-777f-4db3-83d8-e3005a52a620 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:51.495977 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9edba-777f-4db3-83d8-e3005a52a620 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:51.497599 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-eda9edba-777f-4db3-83d8-e3005a52a620 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:51.594076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9edba-777f-4db3-83d8-e3005a52a620 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:51.595720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9edba-777f-4db3-83d8-e3005a52a620 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:51.615343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 681/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:54.056941 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbdae7-ec64-4346-b0ef-34feda1fb2f0 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:54.057454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbdae7-ec64-4346-b0ef-34feda1fb2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:54.057454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbdae7-ec64-4346-b0ef-34feda1fb2f0 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:54.058930 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 696/1377] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:22:54.064197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67064eef-ce25-4890-99e6-981e959fe01f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:54.064591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67064eef-ce25-4890-99e6-981e959fe01f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:54.064777 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67064eef-ce25-4890-99e6-981e959fe01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:54.134428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-67064eef-ce25-4890-99e6-981e959fe01f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:54.135660 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-67064eef-ce25-4890-99e6-981e959fe01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:54.143209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:54.246229 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 682/1378] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:54.539951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbdae7-ec64-4346-b0ef-34feda1fb2f0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:54.541571 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4fbbdae7-ec64-4346-b0ef-34feda1fb2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:54.702294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbdae7-ec64-4346-b0ef-34feda1fb2f0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:54.706555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:54.839147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5cfcb-9ec4-4585-94ce-05a514538008 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:54.839759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5cfcb-9ec4-4585-94ce-05a514538008 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:54.840089 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5cfcb-9ec4-4585-94ce-05a514538008 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:54.840364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5cfcb-9ec4-4585-94ce-05a514538008 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:54.841963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3eb5cfcb-9ec4-4585-94ce-05a514538008 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:54.940495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5cfcb-9ec4-4585-94ce-05a514538008 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:54.942102 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5cfcb-9ec4-4585-94ce-05a514538008 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:54.963678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 697/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:22:57.202291 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba6ce4-173e-4696-8fe5-1fb695d4c7f4 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:57.202840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba6ce4-173e-4696-8fe5-1fb695d4c7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:57.202840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba6ce4-173e-4696-8fe5-1fb695d4c7f4 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:57.204263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 683/1380] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:22:57.209907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d681f-e82c-4ffd-add8-6d1be9d25c3d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:57.210425 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d681f-e82c-4ffd-add8-6d1be9d25c3d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:57.210733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d681f-e82c-4ffd-add8-6d1be9d25c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:57.222728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:57.290352 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ff0d681f-e82c-4ffd-add8-6d1be9d25c3d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:22:57.292289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ff0d681f-e82c-4ffd-add8-6d1be9d25c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:57.408187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 698/1381] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:22:57.745249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba6ce4-173e-4696-8fe5-1fb695d4c7f4 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:22:57.746999 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f4ba6ce4-173e-4696-8fe5-1fb695d4c7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:57.912318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:57.925773 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba6ce4-173e-4696-8fe5-1fb695d4c7f4 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:22:58.061099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f41995-7698-4fe3-b73f-0856b0363793 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:22:58.061735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f41995-7698-4fe3-b73f-0856b0363793 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:22:58.062060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f41995-7698-4fe3-b73f-0856b0363793 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:22:58.062384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f41995-7698-4fe3-b73f-0856b0363793 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:22:58.064371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-94f41995-7698-4fe3-b73f-0856b0363793 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:22:58.170934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f41995-7698-4fe3-b73f-0856b0363793 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:22:58.172902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f41995-7698-4fe3-b73f-0856b0363793 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:22:58.186116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:22:58.196763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 684/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:00.698787 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5fd56-cde9-459e-9edc-87b837d6abfa None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:00.699342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5fd56-cde9-459e-9edc-87b837d6abfa None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:00.699342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5fd56-cde9-459e-9edc-87b837d6abfa None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:00.700778 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 699/1383] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:00.706885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd59cd-0551-44bc-92aa-de80371ca96f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:00.707445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd59cd-0551-44bc-92aa-de80371ca96f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:00.707562 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd59cd-0551-44bc-92aa-de80371ca96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:00.788163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b8cd59cd-0551-44bc-92aa-de80371ca96f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:00.790157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b8cd59cd-0551-44bc-92aa-de80371ca96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:00.909929 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 685/1384] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:01.215069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5fd56-cde9-459e-9edc-87b837d6abfa None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:01.216980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-04f5fd56-cde9-459e-9edc-87b837d6abfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:01.326765 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:01.390825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5fd56-cde9-459e-9edc-87b837d6abfa None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:01.520324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c745bcc-8df8-4c9e-a43b-0279a3695727 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:01.521076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c745bcc-8df8-4c9e-a43b-0279a3695727 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:01.521420 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c745bcc-8df8-4c9e-a43b-0279a3695727 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:01.521756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c745bcc-8df8-4c9e-a43b-0279a3695727 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:01.523973 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9c745bcc-8df8-4c9e-a43b-0279a3695727 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:01.580842 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:01.629515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c745bcc-8df8-4c9e-a43b-0279a3695727 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:01.632794 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c745bcc-8df8-4c9e-a43b-0279a3695727 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:01.656841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 700/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:04.000576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf9b8a-330a-4b4c-bc36-a65adf5ad43e None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:04.001138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf9b8a-330a-4b4c-bc36-a65adf5ad43e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:04.001138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf9b8a-330a-4b4c-bc36-a65adf5ad43e None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:04.002457 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 686/1386] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:04.008278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3bf87-69b7-4fe7-a8eb-78f6c7dacc81 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:04.008693 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3bf87-69b7-4fe7-a8eb-78f6c7dacc81 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:04.008873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3bf87-69b7-4fe7-a8eb-78f6c7dacc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:04.074939 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d2c3bf87-69b7-4fe7-a8eb-78f6c7dacc81 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:04.076160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d2c3bf87-69b7-4fe7-a8eb-78f6c7dacc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:04.173283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 701/1387] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:04.464756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf9b8a-330a-4b4c-bc36-a65adf5ad43e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:04.466319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cbbf9b8a-330a-4b4c-bc36-a65adf5ad43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:04.544206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:04.615706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf9b8a-330a-4b4c-bc36-a65adf5ad43e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:04.732359 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc64a5-8fe4-4c04-a421-329851dcd145 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:04.732912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc64a5-8fe4-4c04-a421-329851dcd145 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:04.733191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc64a5-8fe4-4c04-a421-329851dcd145 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:04.733612 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc64a5-8fe4-4c04-a421-329851dcd145 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:04.735371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5dfc64a5-8fe4-4c04-a421-329851dcd145 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:04.780245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:04.835962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc64a5-8fe4-4c04-a421-329851dcd145 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:04.837941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc64a5-8fe4-4c04-a421-329851dcd145 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:04.855462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57656e42-db8b-4680-a97e-bfb261e67973 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:04.855746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57656e42-db8b-4680-a97e-bfb261e67973 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:04.856052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57656e42-db8b-4680-a97e-bfb261e67973 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:04.856633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 687/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:04.857451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 702/1389] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:23:04.863212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b31831-9032-47ba-ab26-fc1b3e45c59c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:04.863212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b31831-9032-47ba-ab26-fc1b3e45c59c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:04.863422 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b31831-9032-47ba-ab26-fc1b3e45c59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:04.933569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a3b31831-9032-47ba-ab26-fc1b3e45c59c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:04.934881 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a3b31831-9032-47ba-ab26-fc1b3e45c59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:05.044669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 688/1390] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:05.344416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348b283-39b0-402d-8075-c215bc7d2933 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:05.344880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348b283-39b0-402d-8075-c215bc7d2933 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:05.344977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348b283-39b0-402d-8075-c215bc7d2933 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:05.346364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 703/1391] 10.222.0.33 () {54 vars in 966 bytes} [Sun Sep 11 05:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:05.352789 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38011401-3a83-4507-a579-4707059ca612 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:05.353144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38011401-3a83-4507-a579-4707059ca612 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:05.353484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38011401-3a83-4507-a579-4707059ca612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:05.371567 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:05.435260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-38011401-3a83-4507-a579-4707059ca612 None None] MFA Rules not processed for user `0330c79c391d463b91f99e030c6dcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:05.436484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-38011401-3a83-4507-a579-4707059ca612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:05.519808 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 689/1392] 10.222.0.33 () {58 vars in 1063 bytes} [Sun Sep 11 05:23:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:05.531249 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348b283-39b0-402d-8075-c215bc7d2933 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:05.532837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2348b283-39b0-402d-8075-c215bc7d2933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:05.567545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:05.660346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348b283-39b0-402d-8075-c215bc7d2933 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:05.746402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c533405-c5e3-488c-9c3d-ee7e92fdeb09 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0330c79c391d463b91f99e030c6dcc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:05.747011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c533405-c5e3-488c-9c3d-ee7e92fdeb09 service glance] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:05.747285 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c533405-c5e3-488c-9c3d-ee7e92fdeb09 service glance] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:05.747554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c533405-c5e3-488c-9c3d-ee7e92fdeb09 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:05.749168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8c533405-c5e3-488c-9c3d-ee7e92fdeb09 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:05.840861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:05.873558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c533405-c5e3-488c-9c3d-ee7e92fdeb09 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:05.874994 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c533405-c5e3-488c-9c3d-ee7e92fdeb09 service glance] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:05.896542 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 704/1393] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:06.655460 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32dcd4-b09d-4530-b553-29542fcdd10a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:06.655949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32dcd4-b09d-4530-b553-29542fcdd10a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:06.656090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32dcd4-b09d-4530-b553-29542fcdd10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:06.725020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-eb32dcd4-b09d-4530-b553-29542fcdd10a None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:06.726300 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-eb32dcd4-b09d-4530-b553-29542fcdd10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:06.784573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:06.822644 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 690/1394] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 05:23:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:06.867353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae0cfb-22b5-4de9-8ddf-9d614d4330b7 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:06.867847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae0cfb-22b5-4de9-8ddf-9d614d4330b7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:06.867958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae0cfb-22b5-4de9-8ddf-9d614d4330b7 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:06.869353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 705/1395] 10.222.0.33 () {54 vars in 943 bytes} [Sun Sep 11 05:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:06.875937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510e1c1-e2a7-4dcb-8473-13bb7e9f0158 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:06.876263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510e1c1-e2a7-4dcb-8473-13bb7e9f0158 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:06.876543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510e1c1-e2a7-4dcb-8473-13bb7e9f0158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:06.954560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e510e1c1-e2a7-4dcb-8473-13bb7e9f0158 None None] MFA Rules not processed for user `0a726949e8cc4cfdb25aa78f2761cff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:06.955767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e510e1c1-e2a7-4dcb-8473-13bb7e9f0158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:07.070045 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 691/1396] 10.222.0.33 () {58 vars in 1040 bytes} [Sun Sep 11 05:23:06 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:07.079521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae0cfb-22b5-4de9-8ddf-9d614d4330b7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:07.081165 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-28ae0cfb-22b5-4de9-8ddf-9d614d4330b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:07.098240 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3855f-5a15-42de-b283-7d77993df14d None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:07.098478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3855f-5a15-42de-b283-7d77993df14d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:07.098876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3855f-5a15-42de-b283-7d77993df14d None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:07.100255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 692/1397] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:23:07.106752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75953ed8-4ba9-4302-8fb2-2f650378bef0 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:07.107037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75953ed8-4ba9-4302-8fb2-2f650378bef0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:07.107312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75953ed8-4ba9-4302-8fb2-2f650378bef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:07.144175 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:07.174584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-75953ed8-4ba9-4302-8fb2-2f650378bef0 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:07.175843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-75953ed8-4ba9-4302-8fb2-2f650378bef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:07.218178 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae0cfb-22b5-4de9-8ddf-9d614d4330b7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:07.268182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:07.273966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 693/1398] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:23:07.323357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd35683-570a-4024-9556-cb686b825bec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:07.323963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd35683-570a-4024-9556-cb686b825bec service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:07.324267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd35683-570a-4024-9556-cb686b825bec service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:07.324542 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd35683-570a-4024-9556-cb686b825bec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:07.326159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cbd35683-570a-4024-9556-cb686b825bec service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:07.408922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd35683-570a-4024-9556-cb686b825bec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:07.410450 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd35683-570a-4024-9556-cb686b825bec service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:07.418623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 706/1399] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:23:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 05:23:07.575537 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75953ed8-4ba9-4302-8fb2-2f650378bef0 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:07.576917 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-75953ed8-4ba9-4302-8fb2-2f650378bef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:07.702491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:07.708132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75953ed8-4ba9-4302-8fb2-2f650378bef0 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:07.841343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31388e0-2fbd-4a6d-bfbe-9a368c020496 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:07.842063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31388e0-2fbd-4a6d-bfbe-9a368c020496 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:07.842349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31388e0-2fbd-4a6d-bfbe-9a368c020496 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:07.842625 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31388e0-2fbd-4a6d-bfbe-9a368c020496 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:07.844288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d31388e0-2fbd-4a6d-bfbe-9a368c020496 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:07.949089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31388e0-2fbd-4a6d-bfbe-9a368c020496 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:07.951110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31388e0-2fbd-4a6d-bfbe-9a368c020496 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:07.982174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 694/1400] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:10.233735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12718821-7375-498e-b93d-b8227d201747 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:10.234347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12718821-7375-498e-b93d-b8227d201747 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:10.234347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12718821-7375-498e-b93d-b8227d201747 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:10.235813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 707/1401] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:10.241278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3277f58-8c2e-49f4-ac22-c0885c875fb2 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:10.241760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3277f58-8c2e-49f4-ac22-c0885c875fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:10.241978 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3277f58-8c2e-49f4-ac22-c0885c875fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:10.246584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:10.316675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e3277f58-8c2e-49f4-ac22-c0885c875fb2 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:10.317926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e3277f58-8c2e-49f4-ac22-c0885c875fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:10.430607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 695/1402] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:23:10.722892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12718821-7375-498e-b93d-b8227d201747 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:10.724444 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-12718821-7375-498e-b93d-b8227d201747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:10.816590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:10.891659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12718821-7375-498e-b93d-b8227d201747 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:11.024281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e650ce9-d1f5-4dd6-86b7-a330627d2d7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:11.024816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e650ce9-d1f5-4dd6-86b7-a330627d2d7c service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:11.025095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e650ce9-d1f5-4dd6-86b7-a330627d2d7c service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:11.025369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e650ce9-d1f5-4dd6-86b7-a330627d2d7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:11.027004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5e650ce9-d1f5-4dd6-86b7-a330627d2d7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:11.076677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:11.130434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e650ce9-d1f5-4dd6-86b7-a330627d2d7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:11.132716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e650ce9-d1f5-4dd6-86b7-a330627d2d7c service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:11.152806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 708/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:13.389446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ce779-ae66-445a-bcdb-a7fb249e3972 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:13.390029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ce779-ae66-445a-bcdb-a7fb249e3972 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:13.390029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ce779-ae66-445a-bcdb-a7fb249e3972 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:13.391360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 696/1404] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:13.396758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e4a30-ee36-4c1f-9ba5-df0dc59a10b5 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:13.397592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e4a30-ee36-4c1f-9ba5-df0dc59a10b5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:13.398081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e4a30-ee36-4c1f-9ba5-df0dc59a10b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:13.470298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d98e4a30-ee36-4c1f-9ba5-df0dc59a10b5 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:13.471510 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d98e4a30-ee36-4c1f-9ba5-df0dc59a10b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:13.589969 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 709/1405] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:13.897030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ce779-ae66-445a-bcdb-a7fb249e3972 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:13.898991 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a80ce779-ae66-445a-bcdb-a7fb249e3972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:13.997827 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:14.069377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ce779-ae66-445a-bcdb-a7fb249e3972 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:14.198953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839cdcc-8822-4294-8c17-0d85dcf00a85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:14.199695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839cdcc-8822-4294-8c17-0d85dcf00a85 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:14.199977 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839cdcc-8822-4294-8c17-0d85dcf00a85 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:14.200270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839cdcc-8822-4294-8c17-0d85dcf00a85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:14.201872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6839cdcc-8822-4294-8c17-0d85dcf00a85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:14.250943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:14.313582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839cdcc-8822-4294-8c17-0d85dcf00a85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:14.315197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839cdcc-8822-4294-8c17-0d85dcf00a85 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:14.335799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 697/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:16.645861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d795418-ef11-46e2-8c8f-c584e0a2111a None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:16.646464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d795418-ef11-46e2-8c8f-c584e0a2111a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:16.646464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d795418-ef11-46e2-8c8f-c584e0a2111a None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:16.648017 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 710/1407] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:16.653645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a29751-2d6b-4ef9-b28a-925d41d9bfb3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:16.654082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a29751-2d6b-4ef9-b28a-925d41d9bfb3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:16.654394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a29751-2d6b-4ef9-b28a-925d41d9bfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:16.730032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f4a29751-2d6b-4ef9-b28a-925d41d9bfb3 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:16.731593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f4a29751-2d6b-4ef9-b28a-925d41d9bfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:16.844408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 698/1408] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:17.137599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d795418-ef11-46e2-8c8f-c584e0a2111a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:17.139125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1d795418-ef11-46e2-8c8f-c584e0a2111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:17.163728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:17.296101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d795418-ef11-46e2-8c8f-c584e0a2111a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:17.397262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:17.420588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74be403-01fe-48b7-ab03-cb0af7ec53a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:17.421254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74be403-01fe-48b7-ab03-cb0af7ec53a8 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:17.421719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74be403-01fe-48b7-ab03-cb0af7ec53a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:17.421997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74be403-01fe-48b7-ab03-cb0af7ec53a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:17.423626 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c74be403-01fe-48b7-ab03-cb0af7ec53a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:17.530317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74be403-01fe-48b7-ab03-cb0af7ec53a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:17.531919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74be403-01fe-48b7-ab03-cb0af7ec53a8 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:17.551569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 711/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:26.842659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:27.548976 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=53 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:30.198275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e782182-8a1b-40f8-aa9a-72415f819c79 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:30.198844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e782182-8a1b-40f8-aa9a-72415f819c79 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:30.198844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e782182-8a1b-40f8-aa9a-72415f819c79 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:30.200226 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 699/1410] 10.222.0.33 () {54 vars in 918 bytes} [Sun Sep 11 05:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:30.205273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47661e-f0d4-4b57-a1be-93b677fa78e8 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:30.205819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47661e-f0d4-4b57-a1be-93b677fa78e8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:30.205819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47661e-f0d4-4b57-a1be-93b677fa78e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:30.281865 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8b47661e-f0d4-4b57-a1be-93b677fa78e8 None None] MFA Rules not processed for user `4e985df8ad6c4bc7b3cd1328924e3ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:30.283081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8b47661e-f0d4-4b57-a1be-93b677fa78e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:30.390436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 712/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Sun Sep 11 05:23:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:30.405434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d137dc7-d160-4108-beb3-10943b34ee51 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:30.405718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d137dc7-d160-4108-beb3-10943b34ee51 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:30.405989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d137dc7-d160-4108-beb3-10943b34ee51 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:30.407316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 700/1412] 10.222.0.33 () {54 vars in 918 bytes} [Sun Sep 11 05:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:30.412193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d1e34-a26c-4b77-be9d-116a86c24c51 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:30.412541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d1e34-a26c-4b77-be9d-116a86c24c51 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:30.412817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d1e34-a26c-4b77-be9d-116a86c24c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:30.486387 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:30.491121 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-431d1e34-a26c-4b77-be9d-116a86c24c51 None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:30.493008 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-431d1e34-a26c-4b77-be9d-116a86c24c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:30.606653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 713/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Sun Sep 11 05:23:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:33.192789 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64969929-df77-4f84-bb5c-a5d368bdfa31 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:33.194003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64969929-df77-4f84-bb5c-a5d368bdfa31 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:33.194117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64969929-df77-4f84-bb5c-a5d368bdfa31 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:33.195623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:33.200864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5851b48-91f9-4a92-807b-3a1c146b4f61 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:33.201288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5851b48-91f9-4a92-807b-3a1c146b4f61 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:33.201421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5851b48-91f9-4a92-807b-3a1c146b4f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:33.266097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b5851b48-91f9-4a92-807b-3a1c146b4f61 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:33.267313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b5851b48-91f9-4a92-807b-3a1c146b4f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:33.321683 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:33.362970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:33.660805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64969929-df77-4f84-bb5c-a5d368bdfa31 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:33.662404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-64969929-df77-4f84-bb5c-a5d368bdfa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:33.808407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64969929-df77-4f84-bb5c-a5d368bdfa31 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:33.881300 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:33.921058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb9305-cd65-4b79-89e4-0fb16af8bf85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:33.922212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb9305-cd65-4b79-89e4-0fb16af8bf85 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:33.922498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb9305-cd65-4b79-89e4-0fb16af8bf85 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:33.922775 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb9305-cd65-4b79-89e4-0fb16af8bf85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:33.924470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-61cb9305-cd65-4b79-89e4-0fb16af8bf85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:34.014108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb9305-cd65-4b79-89e4-0fb16af8bf85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:34.015716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb9305-cd65-4b79-89e4-0fb16af8bf85 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:34.033416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 702/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:35.173500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6397e92-291c-4d73-a436-79f5eb7e0a29 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:35.174101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6397e92-291c-4d73-a436-79f5eb7e0a29 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:35.174101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6397e92-291c-4d73-a436-79f5eb7e0a29 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:35.175377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 05:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:35.182354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f601d2c-96c7-4960-8d6c-9f9e570b845a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:35.182765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f601d2c-96c7-4960-8d6c-9f9e570b845a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:35.182931 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f601d2c-96c7-4960-8d6c-9f9e570b845a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:35.260181 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-2f601d2c-96c7-4960-8d6c-9f9e570b845a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:35.261375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2f601d2c-96c7-4960-8d6c-9f9e570b845a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:35.288072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:35.357036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 05:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:35.511017 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6397e92-291c-4d73-a436-79f5eb7e0a29 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:35.512506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f6397e92-291c-4d73-a436-79f5eb7e0a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:35.654222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6397e92-291c-4d73-a436-79f5eb7e0a29 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:35.694385 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:35.766551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7247d3-b2b0-44a9-8445-a3a411cc1c1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:35.767072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7247d3-b2b0-44a9-8445-a3a411cc1c1c service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:35.767349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7247d3-b2b0-44a9-8445-a3a411cc1c1c service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:35.767617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7247d3-b2b0-44a9-8445-a3a411cc1c1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:35.769289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ee7247d3-b2b0-44a9-8445-a3a411cc1c1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:35.861297 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7247d3-b2b0-44a9-8445-a3a411cc1c1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:35.862894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7247d3-b2b0-44a9-8445-a3a411cc1c1c service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:35.880774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 716/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:37.019487 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16cb07-42d6-45cc-91b4-4432aa845c2e None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:37.020089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16cb07-42d6-45cc-91b4-4432aa845c2e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:37.020089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16cb07-42d6-45cc-91b4-4432aa845c2e None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:37.022121 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 05:23:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:37.028331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dd44a-a9da-448d-9159-441bf226666c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:37.028781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dd44a-a9da-448d-9159-441bf226666c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:37.028889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dd44a-a9da-448d-9159-441bf226666c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:37.082134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:37.104620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-408dd44a-a9da-448d-9159-441bf226666c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:37.105811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-408dd44a-a9da-448d-9159-441bf226666c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:37.200745 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 05:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:37.352055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16cb07-42d6-45cc-91b4-4432aa845c2e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:37.353486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fc16cb07-42d6-45cc-91b4-4432aa845c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:37.502107 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16cb07-42d6-45cc-91b4-4432aa845c2e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:37.515192 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:37.615365 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9727d7-8410-43e2-8388-8e1da7553c02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:37.616091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9727d7-8410-43e2-8388-8e1da7553c02 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:37.616371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9727d7-8410-43e2-8388-8e1da7553c02 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:37.616639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9727d7-8410-43e2-8388-8e1da7553c02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:37.618228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ed9727d7-8410-43e2-8388-8e1da7553c02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:37.708384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9727d7-8410-43e2-8388-8e1da7553c02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:37.710210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9727d7-8410-43e2-8388-8e1da7553c02 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:37.728642 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 705/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:38.846716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d243b4-d6a2-4909-a8f0-800f4d65ac67 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:38.847243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d243b4-d6a2-4909-a8f0-800f4d65ac67 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:38.847243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d243b4-d6a2-4909-a8f0-800f4d65ac67 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:38.848632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 05:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:38.855217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5dae98-4916-4c74-8c3c-57be21dfb9dd None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:38.855665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5dae98-4916-4c74-8c3c-57be21dfb9dd None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:38.855762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5dae98-4916-4c74-8c3c-57be21dfb9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:38.875977 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:38.932287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-cd5dae98-4916-4c74-8c3c-57be21dfb9dd None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:38.933506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cd5dae98-4916-4c74-8c3c-57be21dfb9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:39.029750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 05:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:39.181663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d243b4-d6a2-4909-a8f0-800f4d65ac67 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:39.183220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f5d243b4-d6a2-4909-a8f0-800f4d65ac67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:39.290714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:39.326633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d243b4-d6a2-4909-a8f0-800f4d65ac67 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:39.438984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a0bc7-5c75-43db-ae37-aab7402ab75e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:39.439508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a0bc7-5c75-43db-ae37-aab7402ab75e service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:39.439784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a0bc7-5c75-43db-ae37-aab7402ab75e service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:39.440082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a0bc7-5c75-43db-ae37-aab7402ab75e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:39.441678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-074a0bc7-5c75-43db-ae37-aab7402ab75e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:39.516655 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:39.530753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a0bc7-5c75-43db-ae37-aab7402ab75e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:39.532365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a0bc7-5c75-43db-ae37-aab7402ab75e service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:39.549880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 719/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:40.663220 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fa812-8c09-4d60-af17-52855a41519c None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:40.663845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fa812-8c09-4d60-af17-52855a41519c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:40.663845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fa812-8c09-4d60-af17-52855a41519c None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:40.665130 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 05:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:40.671385 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b55aeb-d9f2-44a9-abe2-d4b1f866c4c1 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:40.671852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b55aeb-d9f2-44a9-abe2-d4b1f866c4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:40.671957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b55aeb-d9f2-44a9-abe2-d4b1f866c4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:40.748917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-41b55aeb-d9f2-44a9-abe2-d4b1f866c4c1 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:40.750111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-41b55aeb-d9f2-44a9-abe2-d4b1f866c4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:40.846393 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 05:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:40.999007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fa812-8c09-4d60-af17-52855a41519c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:41.000496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bc5fa812-8c09-4d60-af17-52855a41519c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:41.085499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:41.146516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fa812-8c09-4d60-af17-52855a41519c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:41.258134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c269070-5ff4-4894-89a8-f2e929844c7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:41.258657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c269070-5ff4-4894-89a8-f2e929844c7a service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:41.258932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c269070-5ff4-4894-89a8-f2e929844c7a service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:41.259202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c269070-5ff4-4894-89a8-f2e929844c7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:41.260804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6c269070-5ff4-4894-89a8-f2e929844c7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:41.309540 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:41.353171 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c269070-5ff4-4894-89a8-f2e929844c7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:41.354785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c269070-5ff4-4894-89a8-f2e929844c7a service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:41.372899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 708/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:42.435435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7e46b-c148-4706-a426-ec90c6390fdc None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:42.435983 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7e46b-c148-4706-a426-ec90c6390fdc None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:42.435983 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7e46b-c148-4706-a426-ec90c6390fdc None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:42.437499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 05:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:42.444693 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49c461-ebb8-4c35-bcf2-f0ce074ebb5f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:42.445150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49c461-ebb8-4c35-bcf2-f0ce074ebb5f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:42.445252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49c461-ebb8-4c35-bcf2-f0ce074ebb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:42.520187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8b49c461-ebb8-4c35-bcf2-f0ce074ebb5f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:42.521955 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8b49c461-ebb8-4c35-bcf2-f0ce074ebb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:42.619353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 05:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:42.774102 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7e46b-c148-4706-a426-ec90c6390fdc None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:42.775609 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d5d7e46b-c148-4706-a426-ec90c6390fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:42.826313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:42.922999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7e46b-c148-4706-a426-ec90c6390fdc None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:43.034637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55392bdc-8dfc-4ddc-aa53-74f216022b64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:43.035234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55392bdc-8dfc-4ddc-aa53-74f216022b64 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:43.035511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55392bdc-8dfc-4ddc-aa53-74f216022b64 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:43.035780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55392bdc-8dfc-4ddc-aa53-74f216022b64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:43.037950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-55392bdc-8dfc-4ddc-aa53-74f216022b64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:43.044722 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:43.125467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55392bdc-8dfc-4ddc-aa53-74f216022b64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:43.127059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55392bdc-8dfc-4ddc-aa53-74f216022b64 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:43.145206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 722/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:44.200266 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaf957-ec2f-4a9b-a1bb-5a07b3183eba None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:44.200859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaf957-ec2f-4a9b-a1bb-5a07b3183eba None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:44.200859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaf957-ec2f-4a9b-a1bb-5a07b3183eba None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:44.202148 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 05:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:44.208243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f15c87-738d-4e1c-83c9-cfde234f3420 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:44.209235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f15c87-738d-4e1c-83c9-cfde234f3420 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:44.209402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f15c87-738d-4e1c-83c9-cfde234f3420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:44.284448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-96f15c87-738d-4e1c-83c9-cfde234f3420 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:44.285661 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-96f15c87-738d-4e1c-83c9-cfde234f3420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:44.354062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:44.383025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 05:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:44.543736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befaf957-ec2f-4a9b-a1bb-5a07b3183eba None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:44.545205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-befaf957-ec2f-4a9b-a1bb-5a07b3183eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:44.562394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:44.691393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befaf957-ec2f-4a9b-a1bb-5a07b3183eba None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:44.783576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:44.814589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cf5ee-c2b6-4aeb-ba79-7c6f5e044bbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:44.815204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cf5ee-c2b6-4aeb-ba79-7c6f5e044bbe service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:44.815600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cf5ee-c2b6-4aeb-ba79-7c6f5e044bbe service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:44.815874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cf5ee-c2b6-4aeb-ba79-7c6f5e044bbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:44.817523 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d36cf5ee-c2b6-4aeb-ba79-7c6f5e044bbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:44.920515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36cf5ee-c2b6-4aeb-ba79-7c6f5e044bbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:44.922144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36cf5ee-c2b6-4aeb-ba79-7c6f5e044bbe service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:44.942316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 711/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:46.170848 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455afa5-a199-41a7-9625-89a02f42ba4b None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:46.171443 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455afa5-a199-41a7-9625-89a02f42ba4b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:46.171443 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455afa5-a199-41a7-9625-89a02f42ba4b None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:46.172748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 05:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:46.179044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd9680-b1c3-4b42-8a37-356039f734cc None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:46.179562 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd9680-b1c3-4b42-8a37-356039f734cc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:46.179562 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd9680-b1c3-4b42-8a37-356039f734cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:46.253120 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-13cd9680-b1c3-4b42-8a37-356039f734cc None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:46.254310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-13cd9680-b1c3-4b42-8a37-356039f734cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:46.295891 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:46.351813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 05:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:46.503162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455afa5-a199-41a7-9625-89a02f42ba4b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:46.504620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9455afa5-a199-41a7-9625-89a02f42ba4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:46.650820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455afa5-a199-41a7-9625-89a02f42ba4b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:46.704928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:46.763824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d984b90-ad32-43d8-b582-45fd66338aa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:46.764379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d984b90-ad32-43d8-b582-45fd66338aa8 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:46.764663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d984b90-ad32-43d8-b582-45fd66338aa8 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:46.764932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d984b90-ad32-43d8-b582-45fd66338aa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:46.766549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4d984b90-ad32-43d8-b582-45fd66338aa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:46.856160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d984b90-ad32-43d8-b582-45fd66338aa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:46.857761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d984b90-ad32-43d8-b582-45fd66338aa8 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:46.875723 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 725/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:47.919698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ac83a-4c0f-4673-bfec-2f46f19d13f5 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:47.920330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ac83a-4c0f-4673-bfec-2f46f19d13f5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:47.920330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ac83a-4c0f-4673-bfec-2f46f19d13f5 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:47.921637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 05:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:47.927973 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bf86a-5979-4695-a4b9-cd1e3b42e520 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:47.928437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bf86a-5979-4695-a4b9-cd1e3b42e520 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:47.928557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bf86a-5979-4695-a4b9-cd1e3b42e520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:48.005856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2e9bf86a-5979-4695-a4b9-cd1e3b42e520 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:48.007338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2e9bf86a-5979-4695-a4b9-cd1e3b42e520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:48.009401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:48.104195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 05:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:48.252330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ac83a-4c0f-4673-bfec-2f46f19d13f5 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:48.253851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ce7ac83a-4c0f-4673-bfec-2f46f19d13f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:48.399906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ac83a-4c0f-4673-bfec-2f46f19d13f5 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:48.426536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:48.513277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc4b16-8811-4ebd-9d49-55346083562d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:48.513881 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc4b16-8811-4ebd-9d49-55346083562d service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:48.514157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc4b16-8811-4ebd-9d49-55346083562d service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:48.514426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc4b16-8811-4ebd-9d49-55346083562d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:48.516040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-61cc4b16-8811-4ebd-9d49-55346083562d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:48.611560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc4b16-8811-4ebd-9d49-55346083562d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:48.613195 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc4b16-8811-4ebd-9d49-55346083562d service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:48.631410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 714/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:50.714418 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39788a10-47c9-4150-8294-325b4c1b32d6 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:50.714964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39788a10-47c9-4150-8294-325b4c1b32d6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:50.714964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39788a10-47c9-4150-8294-325b4c1b32d6 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:50.716370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:50.721548 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a986cc1-1790-4293-8756-67c228f40f89 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:50.721976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a986cc1-1790-4293-8756-67c228f40f89 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:50.722099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a986cc1-1790-4293-8756-67c228f40f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:50.763787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:50.790287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3a986cc1-1790-4293-8756-67c228f40f89 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:50.791498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3a986cc1-1790-4293-8756-67c228f40f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:50.889600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:51.182160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39788a10-47c9-4150-8294-325b4c1b32d6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:51.183726 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-39788a10-47c9-4150-8294-325b4c1b32d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:51.285289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39788a10-47c9-4150-8294-325b4c1b32d6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:51.435083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:51.476027 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b26bc-51f2-43ce-a5eb-e6f542f39e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:51.476619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b26bc-51f2-43ce-a5eb-e6f542f39e83 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:51.476895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b26bc-51f2-43ce-a5eb-e6f542f39e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:51.477163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b26bc-51f2-43ce-a5eb-e6f542f39e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:51.478737 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b29b26bc-51f2-43ce-a5eb-e6f542f39e83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:51.585704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b26bc-51f2-43ce-a5eb-e6f542f39e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:51.587135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b26bc-51f2-43ce-a5eb-e6f542f39e83 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:51.605526 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 728/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:23:54.015460 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d87433-e5df-445a-8f4e-fdb2271e1eaf None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:54.016042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d87433-e5df-445a-8f4e-fdb2271e1eaf None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:54.016042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d87433-e5df-445a-8f4e-fdb2271e1eaf None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:54.017374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:54.022763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9d409-a756-4f8e-93bb-5cee7db282d4 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:54.023196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9d409-a756-4f8e-93bb-5cee7db282d4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:54.023308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9d409-a756-4f8e-93bb-5cee7db282d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:54.087987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b2e9d409-a756-4f8e-93bb-5cee7db282d4 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:54.089212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b2e9d409-a756-4f8e-93bb-5cee7db282d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:54.101783 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:54.185402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:54.469932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d87433-e5df-445a-8f4e-fdb2271e1eaf None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:54.471483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c0d87433-e5df-445a-8f4e-fdb2271e1eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:54.574764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d87433-e5df-445a-8f4e-fdb2271e1eaf None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:54.587965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:54.651320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd163f-9b13-4588-91ce-a4bab91ca9ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:54.651920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd163f-9b13-4588-91ce-a4bab91ca9ff service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:54.652236 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd163f-9b13-4588-91ce-a4bab91ca9ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:54.652511 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd163f-9b13-4588-91ce-a4bab91ca9ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:54.654095 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9cbd163f-9b13-4588-91ce-a4bab91ca9ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:54.767919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd163f-9b13-4588-91ce-a4bab91ca9ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:54.769374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd163f-9b13-4588-91ce-a4bab91ca9ff service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:54.787105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 717/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:23:57.246986 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9eee9-9516-4c01-95c1-b02fe8c9fa04 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:57.247553 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9eee9-9516-4c01-95c1-b02fe8c9fa04 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:57.247553 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9eee9-9516-4c01-95c1-b02fe8c9fa04 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:57.248920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:23:57.254452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e238dbc-2ee2-448f-a392-de79f1606c5d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:57.255483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e238dbc-2ee2-448f-a392-de79f1606c5d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:57.255636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e238dbc-2ee2-448f-a392-de79f1606c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:57.303821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:57.321951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7e238dbc-2ee2-448f-a392-de79f1606c5d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:23:57.323169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7e238dbc-2ee2-448f-a392-de79f1606c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:57.420720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:23:57.701217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9eee9-9516-4c01-95c1-b02fe8c9fa04 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:23:57.702747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bae9eee9-9516-4c01-95c1-b02fe8c9fa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:57.806739 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9eee9-9516-4c01-95c1-b02fe8c9fa04 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:23:57.849270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:23:57.884358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085efee2-81c5-4bd0-928f-e5a6a847efbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:23:57.884967 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085efee2-81c5-4bd0-928f-e5a6a847efbd service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:23:57.885242 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085efee2-81c5-4bd0-928f-e5a6a847efbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:23:57.885509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085efee2-81c5-4bd0-928f-e5a6a847efbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:23:57.887124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-085efee2-81c5-4bd0-928f-e5a6a847efbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:23:57.993008 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085efee2-81c5-4bd0-928f-e5a6a847efbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:23:57.994358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085efee2-81c5-4bd0-928f-e5a6a847efbd service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:23:58.013539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 731/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:24:00.299709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008698ee-ef89-4ff4-a64b-ae793927e3b6 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:00.300311 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008698ee-ef89-4ff4-a64b-ae793927e3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:00.300311 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008698ee-ef89-4ff4-a64b-ae793927e3b6 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:00.301692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:00.307466 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb12eb2-e10b-4086-a5fc-0bda9febaf19 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:00.307949 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb12eb2-e10b-4086-a5fc-0bda9febaf19 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:00.308119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb12eb2-e10b-4086-a5fc-0bda9febaf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:00.374141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bfb12eb2-e10b-4086-a5fc-0bda9febaf19 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:00.375350 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bfb12eb2-e10b-4086-a5fc-0bda9febaf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:00.397213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:00.472740 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:00.770813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008698ee-ef89-4ff4-a64b-ae793927e3b6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:00.772645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-008698ee-ef89-4ff4-a64b-ae793927e3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:00.877720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008698ee-ef89-4ff4-a64b-ae793927e3b6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:00.899581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:00.953612 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a3885-de04-4eb1-a638-35efaf3e8ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:00.954215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a3885-de04-4eb1-a638-35efaf3e8ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:00.954495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a3885-de04-4eb1-a638-35efaf3e8ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:00.954777 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a3885-de04-4eb1-a638-35efaf3e8ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:00.956398 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f43a3885-de04-4eb1-a638-35efaf3e8ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:01.066364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a3885-de04-4eb1-a638-35efaf3e8ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:01.068459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a3885-de04-4eb1-a638-35efaf3e8ab2 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:01.086527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 720/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:24:03.300714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004b4aa-1c04-4e9a-81a3-dedaedc2ceb3 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:03.301279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004b4aa-1c04-4e9a-81a3-dedaedc2ceb3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:03.301279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004b4aa-1c04-4e9a-81a3-dedaedc2ceb3 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:03.302761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:03.308020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56548443-b80c-497b-8314-680a8a072c93 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:03.308445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56548443-b80c-497b-8314-680a8a072c93 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:03.308588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56548443-b80c-497b-8314-680a8a072c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:03.372146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:03.375074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-56548443-b80c-497b-8314-680a8a072c93 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:03.376907 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-56548443-b80c-497b-8314-680a8a072c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:03.473572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:03.763863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004b4aa-1c04-4e9a-81a3-dedaedc2ceb3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:03.765426 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3004b4aa-1c04-4e9a-81a3-dedaedc2ceb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:03.872702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004b4aa-1c04-4e9a-81a3-dedaedc2ceb3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:03.923492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:03.947754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3f728-ddf8-4389-9697-e04f6fa40d53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:03.948395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3f728-ddf8-4389-9697-e04f6fa40d53 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:03.948717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3f728-ddf8-4389-9697-e04f6fa40d53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:03.948987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3f728-ddf8-4389-9697-e04f6fa40d53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:03.950581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-68e3f728-ddf8-4389-9697-e04f6fa40d53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:04.058803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3f728-ddf8-4389-9697-e04f6fa40d53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:04.060185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3f728-ddf8-4389-9697-e04f6fa40d53 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:04.078421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 734/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:24:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:24:06.337728 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4e80e-4e8f-40de-b062-8a5d9cfd3ea9 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:06.338267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4e80e-4e8f-40de-b062-8a5d9cfd3ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:06.338267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4e80e-4e8f-40de-b062-8a5d9cfd3ea9 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:06.339655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:06.345033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b2796-23ac-4d0d-a1d9-65844e2ff050 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:06.345448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b2796-23ac-4d0d-a1d9-65844e2ff050 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:06.345590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b2796-23ac-4d0d-a1d9-65844e2ff050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:06.411347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-941b2796-23ac-4d0d-a1d9-65844e2ff050 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:06.412741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-941b2796-23ac-4d0d-a1d9-65844e2ff050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:06.441258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:06.508468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:06.795398 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4e80e-4e8f-40de-b062-8a5d9cfd3ea9 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:06.796971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-02f4e80e-4e8f-40de-b062-8a5d9cfd3ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:06.898416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4e80e-4e8f-40de-b062-8a5d9cfd3ea9 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:06.926226 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:06.971507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028aba0f-dbc6-456c-8bb5-11fbb3362e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:06.972157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028aba0f-dbc6-456c-8bb5-11fbb3362e7f service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:06.972439 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028aba0f-dbc6-456c-8bb5-11fbb3362e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:06.972711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028aba0f-dbc6-456c-8bb5-11fbb3362e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:06.974996 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-028aba0f-dbc6-456c-8bb5-11fbb3362e7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:07.084288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028aba0f-dbc6-456c-8bb5-11fbb3362e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:07.085725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028aba0f-dbc6-456c-8bb5-11fbb3362e7f service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:07.103663 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 723/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:24:09.322134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50385fd3-1a6d-492a-86e8-1b360d794128 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:09.322675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50385fd3-1a6d-492a-86e8-1b360d794128 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:09.322675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50385fd3-1a6d-492a-86e8-1b360d794128 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:09.324054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:09.329460 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652e5ca-fc9b-4db9-9cee-905fd6bdac2e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:09.329868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652e5ca-fc9b-4db9-9cee-905fd6bdac2e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:09.330000 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652e5ca-fc9b-4db9-9cee-905fd6bdac2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:09.397045 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-1652e5ca-fc9b-4db9-9cee-905fd6bdac2e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:09.398625 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1652e5ca-fc9b-4db9-9cee-905fd6bdac2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:09.406858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:09.494163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:09.778209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50385fd3-1a6d-492a-86e8-1b360d794128 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:09.779774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-50385fd3-1a6d-492a-86e8-1b360d794128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:09.887384 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50385fd3-1a6d-492a-86e8-1b360d794128 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:09.944020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:09.962062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd317c-8a8b-439a-a758-d696eb1226cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:09.962686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd317c-8a8b-439a-a758-d696eb1226cf service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:09.962945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd317c-8a8b-439a-a758-d696eb1226cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:09.963217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd317c-8a8b-439a-a758-d696eb1226cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:09.964850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-79cd317c-8a8b-439a-a758-d696eb1226cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:10.073659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd317c-8a8b-439a-a758-d696eb1226cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:10.075069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd317c-8a8b-439a-a758-d696eb1226cf service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:10.092666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 737/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:24:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:24:12.319414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf61c1f-f6c3-4562-8b72-b599ae2abf14 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:12.320052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf61c1f-f6c3-4562-8b72-b599ae2abf14 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:12.320052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf61c1f-f6c3-4562-8b72-b599ae2abf14 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:12.321379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:12.326791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cbd3b-e44d-417e-a87c-b5c497650b21 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:12.327214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cbd3b-e44d-417e-a87c-b5c497650b21 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:12.327424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cbd3b-e44d-417e-a87c-b5c497650b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:12.393005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b11cbd3b-e44d-417e-a87c-b5c497650b21 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:12.394245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b11cbd3b-e44d-417e-a87c-b5c497650b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:12.434076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:12.493447 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:12.787628 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf61c1f-f6c3-4562-8b72-b599ae2abf14 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:12.789209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-acf61c1f-f6c3-4562-8b72-b599ae2abf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:12.891389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf61c1f-f6c3-4562-8b72-b599ae2abf14 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:12.929847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:12.965708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca1d09-f59d-4ebb-bba6-fbaeedd603f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:12.966315 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca1d09-f59d-4ebb-bba6-fbaeedd603f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:12.966585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca1d09-f59d-4ebb-bba6-fbaeedd603f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:12.966856 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca1d09-f59d-4ebb-bba6-fbaeedd603f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:12.968465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c8ca1d09-f59d-4ebb-bba6-fbaeedd603f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:13.082815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca1d09-f59d-4ebb-bba6-fbaeedd603f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:13.084410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca1d09-f59d-4ebb-bba6-fbaeedd603f0 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:13.220236 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 726/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:24:15.387050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be3bcb-cd1c-463d-a9fc-a6550e534575 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:15.387672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be3bcb-cd1c-463d-a9fc-a6550e534575 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:15.387672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be3bcb-cd1c-463d-a9fc-a6550e534575 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:15.389001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:15.394375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111cca8-e752-46b7-89db-18b1e456fede None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:15.394887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111cca8-e752-46b7-89db-18b1e456fede None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:15.395538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111cca8-e752-46b7-89db-18b1e456fede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:15.461155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f111cca8-e752-46b7-89db-18b1e456fede None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:15.462629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f111cca8-e752-46b7-89db-18b1e456fede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:15.480378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:15.556980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:15.844271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be3bcb-cd1c-463d-a9fc-a6550e534575 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:15.845829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-34be3bcb-cd1c-463d-a9fc-a6550e534575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:15.954543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be3bcb-cd1c-463d-a9fc-a6550e534575 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:16.026566 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:16.033369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec18f5e-fb42-4a2b-bbcc-a2d7d339362f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:16.033969 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec18f5e-fb42-4a2b-bbcc-a2d7d339362f service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:16.034242 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec18f5e-fb42-4a2b-bbcc-a2d7d339362f service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:16.034509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec18f5e-fb42-4a2b-bbcc-a2d7d339362f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:16.037092 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-eec18f5e-fb42-4a2b-bbcc-a2d7d339362f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:16.144888 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec18f5e-fb42-4a2b-bbcc-a2d7d339362f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:16.146421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec18f5e-fb42-4a2b-bbcc-a2d7d339362f service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:16.164186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 740/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:24:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:24:18.314008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89696140-e6ee-42ae-bbd6-6012028d4e48 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:18.314576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89696140-e6ee-42ae-bbd6-6012028d4e48 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:18.314576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89696140-e6ee-42ae-bbd6-6012028d4e48 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:18.315912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:18.321779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a7782-45d2-4308-9cda-cb13fc097d91 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:18.322181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a7782-45d2-4308-9cda-cb13fc097d91 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:18.322322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a7782-45d2-4308-9cda-cb13fc097d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:18.387980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-757a7782-45d2-4308-9cda-cb13fc097d91 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:18.389254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-757a7782-45d2-4308-9cda-cb13fc097d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:18.436858 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:18.485215 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:18.774004 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89696140-e6ee-42ae-bbd6-6012028d4e48 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:18.775521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-89696140-e6ee-42ae-bbd6-6012028d4e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:18.879098 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89696140-e6ee-42ae-bbd6-6012028d4e48 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:18.927465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:18.953309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fcf459-e838-421a-9536-9cff1a2459c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:18.953904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fcf459-e838-421a-9536-9cff1a2459c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:18.954178 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fcf459-e838-421a-9536-9cff1a2459c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:18.954443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fcf459-e838-421a-9536-9cff1a2459c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:18.956038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-83fcf459-e838-421a-9536-9cff1a2459c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:19.062609 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fcf459-e838-421a-9536-9cff1a2459c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:19.063962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fcf459-e838-421a-9536-9cff1a2459c6 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:19.081847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 729/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:24:28.483690 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=22 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:29.079598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=66 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:39.025959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a167c-0781-48b2-9374-86e233739c5b None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:39.026518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a167c-0781-48b2-9374-86e233739c5b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:39.026518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a167c-0781-48b2-9374-86e233739c5b None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:39.027843 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 742/1471] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:39.033075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1940cb1-ea89-41ee-b3b2-516334036c0e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:39.033480 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1940cb1-ea89-41ee-b3b2-516334036c0e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:39.033664 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1940cb1-ea89-41ee-b3b2-516334036c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:39.100334 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a1940cb1-ea89-41ee-b3b2-516334036c0e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:39.101528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a1940cb1-ea89-41ee-b3b2-516334036c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:39.197100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 730/1472] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:39.494783 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef71a5-c4b2-42b0-b4bf-4d2a9fe0c238 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:39.495067 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef71a5-c4b2-42b0-b4bf-4d2a9fe0c238 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:39.495340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef71a5-c4b2-42b0-b4bf-4d2a9fe0c238 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:39.496718 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 743/1473] 10.222.0.33 () {54 vars in 966 bytes} [Sun Sep 11 05:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:39.501556 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13176bb-1d71-46da-a1a0-c6075651dea4 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:39.501842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13176bb-1d71-46da-a1a0-c6075651dea4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:39.502112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13176bb-1d71-46da-a1a0-c6075651dea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:39.578343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f13176bb-1d71-46da-a1a0-c6075651dea4 None None] MFA Rules not processed for user `0330c79c391d463b91f99e030c6dcc07`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:39.579552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f13176bb-1d71-46da-a1a0-c6075651dea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:39.601242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:39.656528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 731/1474] 10.222.0.33 () {58 vars in 1063 bytes} [Sun Sep 11 05:24:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:39.664164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef71a5-c4b2-42b0-b4bf-4d2a9fe0c238 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:39.665665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f4ef71a5-c4b2-42b0-b4bf-4d2a9fe0c238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:39.768928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef71a5-c4b2-42b0-b4bf-4d2a9fe0c238 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:39.843877 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe97f90-3b0a-42fc-a76b-c22400091725 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0330c79c391d463b91f99e030c6dcc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:39.844502 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe97f90-3b0a-42fc-a76b-c22400091725 service glance] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:39.844802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe97f90-3b0a-42fc-a76b-c22400091725 service glance] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:39.845090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe97f90-3b0a-42fc-a76b-c22400091725 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:39.846654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fbe97f90-3b0a-42fc-a76b-c22400091725 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:39.901526 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:39.956915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe97f90-3b0a-42fc-a76b-c22400091725 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:39.958453 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe97f90-3b0a-42fc-a76b-c22400091725 service glance] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:39.976924 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 744/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:24:42.230363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329e24b-4754-4965-ad8b-235b24578ac4 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:42.230968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329e24b-4754-4965-ad8b-235b24578ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:42.230968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329e24b-4754-4965-ad8b-235b24578ac4 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:42.232317 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 732/1476] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:42.237606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d08ca-ee38-4564-a6a2-fd1ac69e3657 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:42.238586 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d08ca-ee38-4564-a6a2-fd1ac69e3657 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:42.238868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d08ca-ee38-4564-a6a2-fd1ac69e3657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:42.303958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0e4d08ca-ee38-4564-a6a2-fd1ac69e3657 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:42.305193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0e4d08ca-ee38-4564-a6a2-fd1ac69e3657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:42.402235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 745/1477] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:24:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:42.682773 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329e24b-4754-4965-ad8b-235b24578ac4 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:42.684360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b329e24b-4754-4965-ad8b-235b24578ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:42.809393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329e24b-4754-4965-ad8b-235b24578ac4 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:42.858128 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:42.902704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13553a-e0df-4843-89cf-68f9763d7c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:42.903308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13553a-e0df-4843-89cf-68f9763d7c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:42.903585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13553a-e0df-4843-89cf-68f9763d7c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:42.903853 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13553a-e0df-4843-89cf-68f9763d7c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:42.904521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13553a-e0df-4843-89cf-68f9763d7c6a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:42.906264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-1c13553a-e0df-4843-89cf-68f9763d7c6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:24:42.906264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Sep 11 05:24:42.906264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: Sep 11 05:24:42.907403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13553a-e0df-4843-89cf-68f9763d7c6a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:42.926751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 733/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Sun Sep 11 05:24:42 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:24:45.175914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d4a58-7054-4d03-9bea-1e4d23a60121 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:45.176680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d4a58-7054-4d03-9bea-1e4d23a60121 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:45.176680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d4a58-7054-4d03-9bea-1e4d23a60121 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:45.178784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 746/1479] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:24:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:45.185109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9aa6b-5ded-4388-bce6-f6b766376153 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:45.185584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9aa6b-5ded-4388-bce6-f6b766376153 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:45.185810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9aa6b-5ded-4388-bce6-f6b766376153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:45.264969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-dcd9aa6b-5ded-4388-bce6-f6b766376153 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:45.266185 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-dcd9aa6b-5ded-4388-bce6-f6b766376153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:45.367767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 734/1480] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:45.694211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d4a58-7054-4d03-9bea-1e4d23a60121 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:45.695730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6e9d4a58-7054-4d03-9bea-1e4d23a60121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:45.705374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:45.843487 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d4a58-7054-4d03-9bea-1e4d23a60121 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:45.924383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:45.960482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b259ede-1b60-4110-b373-e5e9a24ee43f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:45.960997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b259ede-1b60-4110-b373-e5e9a24ee43f service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:45.961280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b259ede-1b60-4110-b373-e5e9a24ee43f service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:45.961548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b259ede-1b60-4110-b373-e5e9a24ee43f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:45.963155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0b259ede-1b60-4110-b373-e5e9a24ee43f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:46.054371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b259ede-1b60-4110-b373-e5e9a24ee43f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:46.055977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b259ede-1b60-4110-b373-e5e9a24ee43f service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:46.073537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 747/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:24:48.432433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a921a-7c39-4270-b8a1-4641152781ca None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:48.432998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a921a-7c39-4270-b8a1-4641152781ca None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:48.432998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a921a-7c39-4270-b8a1-4641152781ca None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:48.434312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 735/1482] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:48.439531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad0aae-4c94-439a-8a2b-7a8c658db250 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:48.439931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad0aae-4c94-439a-8a2b-7a8c658db250 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:48.440095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad0aae-4c94-439a-8a2b-7a8c658db250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:48.504802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9cad0aae-4c94-439a-8a2b-7a8c658db250 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:48.506000 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9cad0aae-4c94-439a-8a2b-7a8c658db250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:48.551183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:48.601476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 748/1483] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:48.887235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a921a-7c39-4270-b8a1-4641152781ca None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:48.888813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-067a921a-7c39-4270-b8a1-4641152781ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:48.912171 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:48.993230 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a921a-7c39-4270-b8a1-4641152781ca None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:49.065298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3adb4d-ff4f-4cc3-a0e4-d15b2a86e3ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:49.066016 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3adb4d-ff4f-4cc3-a0e4-d15b2a86e3ac service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:49.066310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3adb4d-ff4f-4cc3-a0e4-d15b2a86e3ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:49.066584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3adb4d-ff4f-4cc3-a0e4-d15b2a86e3ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:49.068199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4a3adb4d-ff4f-4cc3-a0e4-d15b2a86e3ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:49.142878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:49.178160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3adb4d-ff4f-4cc3-a0e4-d15b2a86e3ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:49.179582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3adb4d-ff4f-4cc3-a0e4-d15b2a86e3ac service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:49.197374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 736/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:24:51.414933 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8e8ed-58d3-4637-b5da-bd8c6fa3c36a None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:51.415478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8e8ed-58d3-4637-b5da-bd8c6fa3c36a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:51.415478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8e8ed-58d3-4637-b5da-bd8c6fa3c36a None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:51.416838 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 749/1485] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:51.422044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8527259-7db7-4672-9c4f-b808aaaacd1f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:51.422443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8527259-7db7-4672-9c4f-b808aaaacd1f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:51.422593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8527259-7db7-4672-9c4f-b808aaaacd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:51.489594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c8527259-7db7-4672-9c4f-b808aaaacd1f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:51.490790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c8527259-7db7-4672-9c4f-b808aaaacd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:51.589968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 737/1486] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:51.878605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8e8ed-58d3-4637-b5da-bd8c6fa3c36a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:51.880213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-21c8e8ed-58d3-4637-b5da-bd8c6fa3c36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:51.983174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8e8ed-58d3-4637-b5da-bd8c6fa3c36a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:52.054984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:52.056440 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fb0c2-60b7-4d3b-aa02-513daef008ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:52.057099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fb0c2-60b7-4d3b-aa02-513daef008ed service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:52.057565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fb0c2-60b7-4d3b-aa02-513daef008ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:52.058025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fb0c2-60b7-4d3b-aa02-513daef008ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:52.059835 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-772fb0c2-60b7-4d3b-aa02-513daef008ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:52.167869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fb0c2-60b7-4d3b-aa02-513daef008ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:52.169329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fb0c2-60b7-4d3b-aa02-513daef008ed service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:52.187230 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 750/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:24:54.419497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1140c-e3e1-419d-b497-bf3d0bd11b9d None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:54.420055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1140c-e3e1-419d-b497-bf3d0bd11b9d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:54.420055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1140c-e3e1-419d-b497-bf3d0bd11b9d None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:54.421422 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 738/1488] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:24:54.426767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f323421-bc0b-4ea5-9929-9a334dede81b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:54.427194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f323421-bc0b-4ea5-9929-9a334dede81b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:54.427304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f323421-bc0b-4ea5-9929-9a334dede81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:54.494243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7f323421-bc0b-4ea5-9929-9a334dede81b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:54.495440 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7f323421-bc0b-4ea5-9929-9a334dede81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:54.547903 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:54.592508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 751/1489] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:24:54.884109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1140c-e3e1-419d-b497-bf3d0bd11b9d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:54.885687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d0e1140c-e3e1-419d-b497-bf3d0bd11b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:54.921963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:54.992379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1140c-e3e1-419d-b497-bf3d0bd11b9d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:55.066649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163e25e-2cf0-47f6-ab8e-38a3d1250ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:55.067246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163e25e-2cf0-47f6-ab8e-38a3d1250ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:55.067519 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163e25e-2cf0-47f6-ab8e-38a3d1250ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:55.067786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163e25e-2cf0-47f6-ab8e-38a3d1250ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:55.069380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0163e25e-2cf0-47f6-ab8e-38a3d1250ba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:55.148670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:55.178911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163e25e-2cf0-47f6-ab8e-38a3d1250ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:55.180387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163e25e-2cf0-47f6-ab8e-38a3d1250ba4 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:55.197893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 739/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:24:57.552110 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4892109-9960-4f2f-872f-836bb42866b6 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:57.552709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4892109-9960-4f2f-872f-836bb42866b6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:57.552709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4892109-9960-4f2f-872f-836bb42866b6 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:57.554056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 752/1491] 10.222.0.33 () {54 vars in 927 bytes} [Sun Sep 11 05:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:24:57.559308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4a635-9441-4e13-bcf8-1943f98b8d9f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:57.559701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4a635-9441-4e13-bcf8-1943f98b8d9f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:57.559853 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4a635-9441-4e13-bcf8-1943f98b8d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:57.626032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7aa4a635-9441-4e13-bcf8-1943f98b8d9f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:24:57.627241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7aa4a635-9441-4e13-bcf8-1943f98b8d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:57.722499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 740/1492] 10.222.0.33 () {58 vars in 1024 bytes} [Sun Sep 11 05:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:24:58.015011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4892109-9960-4f2f-872f-836bb42866b6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:24:58.016550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a4892109-9960-4f2f-872f-836bb42866b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:58.119623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4892109-9960-4f2f-872f-836bb42866b6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:24:58.192392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbc403-f79c-49d0-ba5c-204c1a26eda0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:24:58.192993 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbc403-f79c-49d0-ba5c-204c1a26eda0 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:24:58.193274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbc403-f79c-49d0-ba5c-204c1a26eda0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:24:58.193537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbc403-f79c-49d0-ba5c-204c1a26eda0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:24:58.195125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9bcbc403-f79c-49d0-ba5c-204c1a26eda0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:24:58.203528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:24:58.305404 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbc403-f79c-49d0-ba5c-204c1a26eda0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:24:58.306836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbc403-f79c-49d0-ba5c-204c1a26eda0 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:24:58.324801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 753/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:25:07.720745 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=82 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:25:08.322519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=50 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:26:10.757097 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4abb6-112b-4f32-ae0d-09f030b94b20 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:10.757653 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4abb6-112b-4f32-ae0d-09f030b94b20 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:10.757653 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4abb6-112b-4f32-ae0d-09f030b94b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:10.835542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e4d4abb6-112b-4f32-ae0d-09f030b94b20 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:26:10.836805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e4d4abb6-112b-4f32-ae0d-09f030b94b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:10.933440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 741/1494] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:26:10.942561 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17164a20-b9e9-4aa2-8f43-0fd944234eee None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:10.943065 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17164a20-b9e9-4aa2-8f43-0fd944234eee None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:10.943261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17164a20-b9e9-4aa2-8f43-0fd944234eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:11.019303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-17164a20-b9e9-4aa2-8f43-0fd944234eee None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:26:11.020461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-17164a20-b9e9-4aa2-8f43-0fd944234eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:11.116783 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 754/1495] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:26:11.121409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4abb6-112b-4f32-ae0d-09f030b94b20 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:26:11.122631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e4d4abb6-112b-4f32-ae0d-09f030b94b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:11.135103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4abb6-112b-4f32-ae0d-09f030b94b20 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:26:11.196699 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:26:11.230811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fa267-a8d0-4f35-9f08-16e22dad3f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:26:11.231412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fa267-a8d0-4f35-9f08-16e22dad3f33 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:11.231692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fa267-a8d0-4f35-9f08-16e22dad3f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:11.231964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fa267-a8d0-4f35-9f08-16e22dad3f33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:11.232729 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4fa267-a8d0-4f35-9f08-16e22dad3f33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:26:11.233034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4fa267-a8d0-4f35-9f08-16e22dad3f33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:26:11.234053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-8c4fa267-a8d0-4f35-9f08-16e22dad3f33 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:26:11.234053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Sep 11 05:26:11.234053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: Sep 11 05:26:11.235455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4fa267-a8d0-4f35-9f08-16e22dad3f33 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:26:11.246491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 742/1496] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:26:11.255590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17164a20-b9e9-4aa2-8f43-0fd944234eee None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:26:11.265008 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17164a20-b9e9-4aa2-8f43-0fd944234eee None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:26:11.325979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:26:11.358830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fa97a-0632-4a8f-a727-e98262cb83ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:26:11.359435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fa97a-0632-4a8f-a727-e98262cb83ad admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:11.360264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fa97a-0632-4a8f-a727-e98262cb83ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:11.360533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fa97a-0632-4a8f-a727-e98262cb83ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:11.361169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fa97a-0632-4a8f-a727-e98262cb83ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:26:11.362886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-ba6fa97a-0632-4a8f-a727-e98262cb83ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:26:11.362886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 11 05:26:11.362886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: Sep 11 05:26:11.364111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fa97a-0632-4a8f-a727-e98262cb83ad admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:26:11.419543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 755/1497] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:26:11 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:26:11.424337 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fa267-a8d0-4f35-9f08-16e22dad3f33 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:26:11.425854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8c4fa267-a8d0-4f35-9f08-16e22dad3f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:11.459487 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fa267-a8d0-4f35-9f08-16e22dad3f33 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:26:11.552895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36297c3e-e749-453e-a07e-5e28ced2e019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:26:11.553492 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36297c3e-e749-453e-a07e-5e28ced2e019 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:11.553771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36297c3e-e749-453e-a07e-5e28ced2e019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:11.554040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36297c3e-e749-453e-a07e-5e28ced2e019 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:11.554655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36297c3e-e749-453e-a07e-5e28ced2e019 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:26:11.555671 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-36297c3e-e749-453e-a07e-5e28ced2e019 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:26:11.555671 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Sep 11 05:26:11.555671 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: Sep 11 05:26:11.556911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36297c3e-e749-453e-a07e-5e28ced2e019 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:26:11.598517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 743/1498] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:26:11 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:26:11.602632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fa97a-0632-4a8f-a727-e98262cb83ad admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:26:11.604209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ba6fa97a-0632-4a8f-a727-e98262cb83ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:11.671013 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fa97a-0632-4a8f-a727-e98262cb83ad admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:26:11.753540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:26:11.765490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929ad2c-0912-4b1e-b963-a43eaad18ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:26:11.766234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929ad2c-0912-4b1e-b963-a43eaad18ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:11.766735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929ad2c-0912-4b1e-b963-a43eaad18ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:11.767014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929ad2c-0912-4b1e-b963-a43eaad18ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:11.767686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929ad2c-0912-4b1e-b963-a43eaad18ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:26:11.768725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-7929ad2c-0912-4b1e-b963-a43eaad18ef2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:26:11.768725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 11 05:26:11.768725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: Sep 11 05:26:11.769608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929ad2c-0912-4b1e-b963-a43eaad18ef2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:26:11.783535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 756/1499] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:26:11.787581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36297c3e-e749-453e-a07e-5e28ced2e019 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:26:11.789345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-36297c3e-e749-453e-a07e-5e28ced2e019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:11.802211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36297c3e-e749-453e-a07e-5e28ced2e019 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:26:11.827385 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:26:11.898428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dd4e1-c5b2-4dbc-8ec3-a9a595e6209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:26:11.898935 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dd4e1-c5b2-4dbc-8ec3-a9a595e6209e admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:11.899214 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dd4e1-c5b2-4dbc-8ec3-a9a595e6209e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:11.899482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dd4e1-c5b2-4dbc-8ec3-a9a595e6209e admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:11.900147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dd4e1-c5b2-4dbc-8ec3-a9a595e6209e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:26:11.901205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-7f1dd4e1-c5b2-4dbc-8ec3-a9a595e6209e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:26:11.901205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Sep 11 05:26:11.901205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: Sep 11 05:26:11.902090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dd4e1-c5b2-4dbc-8ec3-a9a595e6209e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:26:11.915161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 744/1500] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:26:11.919449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929ad2c-0912-4b1e-b963-a43eaad18ef2 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:26:11.920990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7929ad2c-0912-4b1e-b963-a43eaad18ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:11.955199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929ad2c-0912-4b1e-b963-a43eaad18ef2 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:26:12.048225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47765a0f-cd59-41af-84fa-1703bd29269e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:26:12.048873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47765a0f-cd59-41af-84fa-1703bd29269e admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:12.049153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47765a0f-cd59-41af-84fa-1703bd29269e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:12.049427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47765a0f-cd59-41af-84fa-1703bd29269e admin admin] PATH_INFO: `/v3/projects/4533657424244f9c9ce5b6e631cd951c/users/c630ae12a9224bb4a69a9c7b91f0da07/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:12.071255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47765a0f-cd59-41af-84fa-1703bd29269e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4533657424244f9c9ce5b6e631cd951c, user_id=c630ae12a9224bb4a69a9c7b91f0da07, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:26:12.072306 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-47765a0f-cd59-41af-84fa-1703bd29269e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:26:12.072306 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 11 05:26:12.072306 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: Sep 11 05:26:12.074253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47765a0f-cd59-41af-84fa-1703bd29269e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:26:12.099279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 757/1501] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:26:11 2022] PUT /identity/v3/projects/4533657424244f9c9ce5b6e631cd951c/users/c630ae12a9224bb4a69a9c7b91f0da07/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:26:12.112019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f4a6f-03fb-4d3d-8c2e-75665efe813e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:12.112371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f4a6f-03fb-4d3d-8c2e-75665efe813e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:12.112650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f4a6f-03fb-4d3d-8c2e-75665efe813e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:12.169529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c81f4a6f-03fb-4d3d-8c2e-75665efe813e None None] MFA Rules not processed for user `c630ae12a9224bb4a69a9c7b91f0da07`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:26:12.170665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c81f4a6f-03fb-4d3d-8c2e-75665efe813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:12.247912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 745/1502] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:26:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:26:12.261409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2273-ee31-4621-a851-bbd3f917a954 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:12.261698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2273-ee31-4621-a851-bbd3f917a954 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:12.261976 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2273-ee31-4621-a851-bbd3f917a954 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:12.263316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 758/1503] 10.222.0.33 () {54 vars in 964 bytes} [Sun Sep 11 05:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:26:12.268183 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03c91-b210-41ae-8e0f-c9a742a9cd55 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:12.268469 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03c91-b210-41ae-8e0f-c9a742a9cd55 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:12.268744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03c91-b210-41ae-8e0f-c9a742a9cd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:12.300673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:26:12.345438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-15c03c91-b210-41ae-8e0f-c9a742a9cd55 None None] MFA Rules not processed for user `8ef93155a40f4c69b320ca962d4e8868`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:26:12.346634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-15c03c91-b210-41ae-8e0f-c9a742a9cd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:12.421591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 746/1504] 10.222.0.33 () {58 vars in 1061 bytes} [Sun Sep 11 05:26:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:26:12.431601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b2273-ee31-4621-a851-bbd3f917a954 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:26:12.433206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-474b2273-ee31-4621-a851-bbd3f917a954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:12.486207 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:26:12.539891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b2273-ee31-4621-a851-bbd3f917a954 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:26:12.616964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ac9a7-a3cb-480d-b2b0-77ab60af0732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:26:12.617568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ac9a7-a3cb-480d-b2b0-77ab60af0732 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:12.617843 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ac9a7-a3cb-480d-b2b0-77ab60af0732 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:12.618111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ac9a7-a3cb-480d-b2b0-77ab60af0732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:12.619767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ee9ac9a7-a3cb-480d-b2b0-77ab60af0732 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:12.722732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ac9a7-a3cb-480d-b2b0-77ab60af0732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:26:12.726187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ac9a7-a3cb-480d-b2b0-77ab60af0732 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:26:12.728591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:26:12.742113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 759/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:26:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:26:12.788161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e927e63-988c-405f-8271-4ae16eaf8ef3 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:12.788599 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e927e63-988c-405f-8271-4ae16eaf8ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:12.788860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e927e63-988c-405f-8271-4ae16eaf8ef3 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:12.790201 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 747/1506] 10.222.0.33 () {52 vars in 830 bytes} [Sun Sep 11 05:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:26:12.793982 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ac9a7-a3cb-480d-b2b0-77ab60af0732 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:26:12.795198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ee9ac9a7-a3cb-480d-b2b0-77ab60af0732 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:12.894769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ac9a7-a3cb-480d-b2b0-77ab60af0732 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:26:12.987534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:26:12.988229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:12.988379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:12.988644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] PATH_INFO: `/v3/users/c630ae12a9224bb4a69a9c7b91f0da07` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:13.001890 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c630ae12a9224bb4a69a9c7b91f0da07)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:26:13.002925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:26:13.002925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 11 05:26:13.002925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: Sep 11 05:26:13.004185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:26:13.014225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:26:13.072278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c630ae12a9224bb4a69a9c7b91f0da07'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:26:13.072712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c630ae12a9224bb4a69a9c7b91f0da07'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:26:13.086873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c630ae12a9224bb4a69a9c7b91f0da07'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:26:13.096234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c630ae12a9224bb4a69a9c7b91f0da07'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:26:13.120546 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c630ae12a9224bb4a69a9c7b91f0da07'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:26:13.126869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 760/1507] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:26:12 2022] DELETE /identity/v3/users/c630ae12a9224bb4a69a9c7b91f0da07 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:26:13.131328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d268a-a831-4f65-84a1-aec65b0fac28 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:13.131875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d268a-a831-4f65-84a1-aec65b0fac28 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:13.132192 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d268a-a831-4f65-84a1-aec65b0fac28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:13.189488 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-507d268a-a831-4f65-84a1-aec65b0fac28 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:26:13.190719 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-507d268a-a831-4f65-84a1-aec65b0fac28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:13.265840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:26:13.278871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 748/1508] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:26:13.288623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:26:13.289842 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:13.390402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3d8d1-8ce5-4ada-98ba-b1ada93bc444 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:26:13.445819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:26:13.465511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fee5b9-b28f-4b18-b1b9-19ae7def5ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:26:13.466121 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fee5b9-b28f-4b18-b1b9-19ae7def5ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:13.466400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fee5b9-b28f-4b18-b1b9-19ae7def5ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:13.466671 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fee5b9-b28f-4b18-b1b9-19ae7def5ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:13.468286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-22fee5b9-b28f-4b18-b1b9-19ae7def5ea3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:13.576206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fee5b9-b28f-4b18-b1b9-19ae7def5ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:26:13.577657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fee5b9-b28f-4b18-b1b9-19ae7def5ea3 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:26:13.595670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 761/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:26:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:26:13.924474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d268a-a831-4f65-84a1-aec65b0fac28 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:26:13.925877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-507d268a-a831-4f65-84a1-aec65b0fac28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:26:13.937735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d268a-a831-4f65-84a1-aec65b0fac28 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:26:14.035922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df7490-07af-4dd7-81ba-43570799dce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:26:14.036578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df7490-07af-4dd7-81ba-43570799dce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:26:14.036852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df7490-07af-4dd7-81ba-43570799dce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:26:14.037119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df7490-07af-4dd7-81ba-43570799dce1 admin admin] PATH_INFO: `/v3/projects/4533657424244f9c9ce5b6e631cd951c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:26:14.046809 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df7490-07af-4dd7-81ba-43570799dce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4533657424244f9c9ce5b6e631cd951c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:26:14.048407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-18df7490-07af-4dd7-81ba-43570799dce1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:26:14.048407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Sep 11 05:26:14.048407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: Sep 11 05:26:14.049737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df7490-07af-4dd7-81ba-43570799dce1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:26:14.107381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-18df7490-07af-4dd7-81ba-43570799dce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4533657424244f9c9ce5b6e631cd951c', 'request_id': 'req-18df7490-07af-4dd7-81ba-43570799dce1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:26:14.116545 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-18df7490-07af-4dd7-81ba-43570799dce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4533657424244f9c9ce5b6e631cd951c', 'request_id': 'req-18df7490-07af-4dd7-81ba-43570799dce1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:26:14.130857 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-18df7490-07af-4dd7-81ba-43570799dce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4533657424244f9c9ce5b6e631cd951c', 'request_id': 'req-18df7490-07af-4dd7-81ba-43570799dce1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:26:14.153585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-18df7490-07af-4dd7-81ba-43570799dce1 admin admin] The token cache is being invalidate because project 4533657424244f9c9ce5b6e631cd951c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:26:14.153869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-18df7490-07af-4dd7-81ba-43570799dce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:26:14.154295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-18df7490-07af-4dd7-81ba-43570799dce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:26:14.155643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 749/1510] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:26:13 2022] DELETE /identity/v3/projects/4533657424244f9c9ce5b6e631cd951c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:26:23.593270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=62 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:26:24.151584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:27:40.523747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696422e-b181-4d36-ae87-566069ece3ea None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:40.524353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696422e-b181-4d36-ae87-566069ece3ea None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:40.524353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696422e-b181-4d36-ae87-566069ece3ea None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:40.525684 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 05:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:27:40.530774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48525d-e3e4-4b8d-bfbc-58e8d0543b67 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:40.531223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48525d-e3e4-4b8d-bfbc-58e8d0543b67 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:40.531336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48525d-e3e4-4b8d-bfbc-58e8d0543b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:40.612421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-fa48525d-e3e4-4b8d-bfbc-58e8d0543b67 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:27:40.613633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fa48525d-e3e4-4b8d-bfbc-58e8d0543b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:27:40.716865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 05:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:27:40.871188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696422e-b181-4d36-ae87-566069ece3ea None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:27:40.873328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4696422e-b181-4d36-ae87-566069ece3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:27:40.995470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696422e-b181-4d36-ae87-566069ece3ea None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:27:41.001500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be45cc-c9aa-4509-b14a-d43582daf8e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:27:41.002241 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be45cc-c9aa-4509-b14a-d43582daf8e1 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:41.002522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be45cc-c9aa-4509-b14a-d43582daf8e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:41.002796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be45cc-c9aa-4509-b14a-d43582daf8e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:41.010995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be45cc-c9aa-4509-b14a-d43582daf8e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:27:41.012609 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be45cc-c9aa-4509-b14a-d43582daf8e1 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:27:41.022540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 763/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:27:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:27:42.101585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c600636-bcf6-437e-8452-5d32fe06d20d None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:42.102126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c600636-bcf6-437e-8452-5d32fe06d20d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:42.102126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c600636-bcf6-437e-8452-5d32fe06d20d None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:42.103461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 05:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:27:42.109602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6c173-40b1-4c54-b99a-ebbe74a4b098 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:42.110539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6c173-40b1-4c54-b99a-ebbe74a4b098 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:42.110712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6c173-40b1-4c54-b99a-ebbe74a4b098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:42.130742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-10b6c173-40b1-4c54-b99a-ebbe74a4b098 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:27:42.132259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-10b6c173-40b1-4c54-b99a-ebbe74a4b098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:27:42.140179 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 05:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:27:42.292677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c600636-bcf6-437e-8452-5d32fe06d20d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:27:42.307340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c600636-bcf6-437e-8452-5d32fe06d20d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:27:42.312810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effcf3c-cf33-4495-ac4a-54f49a73012f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:27:42.313408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effcf3c-cf33-4495-ac4a-54f49a73012f service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:42.313687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effcf3c-cf33-4495-ac4a-54f49a73012f service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:42.313957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effcf3c-cf33-4495-ac4a-54f49a73012f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:42.321868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effcf3c-cf33-4495-ac4a-54f49a73012f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:27:42.323488 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8effcf3c-cf33-4495-ac4a-54f49a73012f service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:27:42.332699 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 752/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:27:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:27:42.511691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191d087-8210-426a-aab0-5d984a218924 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:42.511971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191d087-8210-426a-aab0-5d984a218924 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:42.512334 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191d087-8210-426a-aab0-5d984a218924 None None] PATH_INFO: `/v3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:42.513627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 765/1517] 10.222.0.17 () {54 vars in 925 bytes} [Sun Sep 11 05:27:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:27:42.519906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf63b49-af21-44a3-ad4e-0196e5143ece None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:42.520268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf63b49-af21-44a3-ad4e-0196e5143ece None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:42.520565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf63b49-af21-44a3-ad4e-0196e5143ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:42.584633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-6cf63b49-af21-44a3-ad4e-0196e5143ece None None] MFA Rules not processed for user `4e985df8ad6c4bc7b3cd1328924e3ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:27:42.585920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6cf63b49-af21-44a3-ad4e-0196e5143ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:27:42.603297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:27:42.660201 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 753/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Sun Sep 11 05:27:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:27:51.671799 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e40b0-e172-4774-9099-66f5196868a5 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:51.672876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e40b0-e172-4774-9099-66f5196868a5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:51.672992 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e40b0-e172-4774-9099-66f5196868a5 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:51.674234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 766/1519] 10.222.0.33 () {54 vars in 961 bytes} [Sun Sep 11 05:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:27:51.679102 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa85a2c-d735-46cd-8256-f8ecfa162060 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:51.679550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa85a2c-d735-46cd-8256-f8ecfa162060 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:51.679741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa85a2c-d735-46cd-8256-f8ecfa162060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:51.700060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3fa85a2c-d735-46cd-8256-f8ecfa162060 None None] MFA Rules not processed for user `4e985df8ad6c4bc7b3cd1328924e3ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:27:51.701261 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3fa85a2c-d735-46cd-8256-f8ecfa162060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:27:51.709868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 754/1520] 10.222.0.33 () {58 vars in 1058 bytes} [Sun Sep 11 05:27:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:27:51.717467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e40b0-e172-4774-9099-66f5196868a5 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:27:51.730756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e40b0-e172-4774-9099-66f5196868a5 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:27:51.736410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75037a-87eb-49bd-9d48-49781a27db93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:27:51.737188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75037a-87eb-49bd-9d48-49781a27db93 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:51.737472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75037a-87eb-49bd-9d48-49781a27db93 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:51.737745 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75037a-87eb-49bd-9d48-49781a27db93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:51.746162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75037a-87eb-49bd-9d48-49781a27db93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:27:51.747746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75037a-87eb-49bd-9d48-49781a27db93 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:27:51.758353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 767/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:27:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:27:53.821103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3108235-9063-4f52-960b-a5b4d833b3a5 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:53.821552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3108235-9063-4f52-960b-a5b4d833b3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:53.821656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3108235-9063-4f52-960b-a5b4d833b3a5 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:53.823001 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 05:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:27:53.829209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21feb90-3096-435b-be7c-978c3e4063f9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:53.829690 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21feb90-3096-435b-be7c-978c3e4063f9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:53.829797 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21feb90-3096-435b-be7c-978c3e4063f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:53.848857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b21feb90-3096-435b-be7c-978c3e4063f9 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:27:53.850053 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b21feb90-3096-435b-be7c-978c3e4063f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:27:53.857854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 05:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:27:54.016428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3108235-9063-4f52-960b-a5b4d833b3a5 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:27:54.030491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3108235-9063-4f52-960b-a5b4d833b3a5 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:27:54.036686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355b736-2887-4537-99ab-5bff2c219598 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:27:54.037275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355b736-2887-4537-99ab-5bff2c219598 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:54.037552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355b736-2887-4537-99ab-5bff2c219598 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:54.037824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355b736-2887-4537-99ab-5bff2c219598 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:54.045694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355b736-2887-4537-99ab-5bff2c219598 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:27:54.047450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355b736-2887-4537-99ab-5bff2c219598 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:27:54.056880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 756/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:27:55.123781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f689fae-5f89-4a6d-a9a0-8e267cbb2c76 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:55.124374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f689fae-5f89-4a6d-a9a0-8e267cbb2c76 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:55.124374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f689fae-5f89-4a6d-a9a0-8e267cbb2c76 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:55.126606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 05:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:27:55.133319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07502041-a934-40c7-94b3-0b4629f71d85 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:55.133763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07502041-a934-40c7-94b3-0b4629f71d85 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:55.133881 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07502041-a934-40c7-94b3-0b4629f71d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:55.153610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-07502041-a934-40c7-94b3-0b4629f71d85 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:27:55.154810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-07502041-a934-40c7-94b3-0b4629f71d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:27:55.163016 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 05:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:27:55.312452 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f689fae-5f89-4a6d-a9a0-8e267cbb2c76 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:27:55.326647 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f689fae-5f89-4a6d-a9a0-8e267cbb2c76 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:27:55.332859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75193f01-6448-4af8-bddc-4bccf18e2bb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:27:55.333446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75193f01-6448-4af8-bddc-4bccf18e2bb6 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:55.333721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75193f01-6448-4af8-bddc-4bccf18e2bb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:55.333989 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75193f01-6448-4af8-bddc-4bccf18e2bb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:55.341750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75193f01-6448-4af8-bddc-4bccf18e2bb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:27:55.343313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75193f01-6448-4af8-bddc-4bccf18e2bb6 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:27:55.353733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 770/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:27:59.458265 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896d88f-8471-403f-9e65-acbaf962321a None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:59.458815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896d88f-8471-403f-9e65-acbaf962321a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:59.458815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896d88f-8471-403f-9e65-acbaf962321a None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:59.460250 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:27:59.465366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24455a-1d4c-4940-8c49-e6c7ba7503d0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:59.465780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24455a-1d4c-4940-8c49-e6c7ba7503d0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:59.465920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24455a-1d4c-4940-8c49-e6c7ba7503d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:59.486079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1a24455a-1d4c-4940-8c49-e6c7ba7503d0 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:27:59.487281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1a24455a-1d4c-4940-8c49-e6c7ba7503d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:27:59.495066 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:27:59.785330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896d88f-8471-403f-9e65-acbaf962321a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:27:59.787035 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a896d88f-8471-403f-9e65-acbaf962321a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:27:59.866659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896d88f-8471-403f-9e65-acbaf962321a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:27:59.870173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824bde1-38df-49ab-9050-6b7352272314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:27:59.870766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824bde1-38df-49ab-9050-6b7352272314 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:27:59.871041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824bde1-38df-49ab-9050-6b7352272314 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:27:59.871309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824bde1-38df-49ab-9050-6b7352272314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:27:59.880723 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824bde1-38df-49ab-9050-6b7352272314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:27:59.882450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824bde1-38df-49ab-9050-6b7352272314 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:27:59.891907 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 759/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:28:02.050016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dee74-c2fd-4020-b671-2096b3c3271c None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:28:02.050641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dee74-c2fd-4020-b671-2096b3c3271c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:28:02.050641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dee74-c2fd-4020-b671-2096b3c3271c None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:28:02.051905 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:28:02.057250 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146234cd-a333-43b3-9ed8-986322ddb49d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:28:02.057698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146234cd-a333-43b3-9ed8-986322ddb49d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:28:02.057817 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146234cd-a333-43b3-9ed8-986322ddb49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:28:02.077087 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-146234cd-a333-43b3-9ed8-986322ddb49d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:28:02.078267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-146234cd-a333-43b3-9ed8-986322ddb49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:28:02.086697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:28:02.375613 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dee74-c2fd-4020-b671-2096b3c3271c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:28:02.387199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dee74-c2fd-4020-b671-2096b3c3271c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:28:02.390729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cbd87-2416-4ce8-8da2-aa6728459a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:28:02.391322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cbd87-2416-4ce8-8da2-aa6728459a7c service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:28:02.391852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cbd87-2416-4ce8-8da2-aa6728459a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:28:02.392163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cbd87-2416-4ce8-8da2-aa6728459a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:28:02.401608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cbd87-2416-4ce8-8da2-aa6728459a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:28:02.403048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cbd87-2416-4ce8-8da2-aa6728459a7c service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:28:02.413358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 773/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:28:06.252424 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a4037-5733-48c1-a55f-f04fe7e859ab None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:28:06.253000 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a4037-5733-48c1-a55f-f04fe7e859ab None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:28:06.253000 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a4037-5733-48c1-a55f-f04fe7e859ab None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:28:06.254298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:28:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:28:06.259787 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8224c-2870-4013-8350-a0f0e747ce54 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:28:06.260265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8224c-2870-4013-8350-a0f0e747ce54 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:28:06.260401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8224c-2870-4013-8350-a0f0e747ce54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:28:06.268669 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=81 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:28:06.281266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bcc8224c-2870-4013-8350-a0f0e747ce54 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:28:06.282810 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bcc8224c-2870-4013-8350-a0f0e747ce54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:28:06.290591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:28:06.593144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a4037-5733-48c1-a55f-f04fe7e859ab None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:28:06.594877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a21a4037-5733-48c1-a55f-f04fe7e859ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:28:06.659412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:28:06.672792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a4037-5733-48c1-a55f-f04fe7e859ab None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:28:06.676404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baba678-9d6c-4717-899f-e229b2e80714 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:28:06.677006 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baba678-9d6c-4717-899f-e229b2e80714 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:28:06.677284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baba678-9d6c-4717-899f-e229b2e80714 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:28:06.677555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baba678-9d6c-4717-899f-e229b2e80714 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:28:06.686317 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baba678-9d6c-4717-899f-e229b2e80714 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:28:06.687743 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baba678-9d6c-4717-899f-e229b2e80714 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:28:06.697607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 762/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:28:08.837886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbf638-c56d-4bad-a64a-c35660df2525 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:28:08.838434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbf638-c56d-4bad-a64a-c35660df2525 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:28:08.838434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbf638-c56d-4bad-a64a-c35660df2525 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:28:08.839810 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 05:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:28:08.846925 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef23a-3bb6-433e-b94d-6df95347f3c3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:28:08.847367 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef23a-3bb6-433e-b94d-6df95347f3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:28:08.847473 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef23a-3bb6-433e-b94d-6df95347f3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:28:08.866704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b46ef23a-3bb6-433e-b94d-6df95347f3c3 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:28:08.867891 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b46ef23a-3bb6-433e-b94d-6df95347f3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:28:08.875531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 05:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:28:09.180392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbf638-c56d-4bad-a64a-c35660df2525 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:28:09.182054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f2dbf638-c56d-4bad-a64a-c35660df2525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:28:09.196090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbf638-c56d-4bad-a64a-c35660df2525 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:28:09.199518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3e5a7-2912-41cb-99bf-d8ff77952694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:28:09.200203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3e5a7-2912-41cb-99bf-d8ff77952694 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:28:09.200482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3e5a7-2912-41cb-99bf-d8ff77952694 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:28:09.200750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3e5a7-2912-41cb-99bf-d8ff77952694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:28:09.210280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3e5a7-2912-41cb-99bf-d8ff77952694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:28:09.211682 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3e5a7-2912-41cb-99bf-d8ff77952694 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:28:09.221023 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 776/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:28:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:28:18.862023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=11 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:28:19.216693 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=8 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:34.212649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827d72e-aa41-4c80-a044-ac046dfd95fd None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:34.213286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827d72e-aa41-4c80-a044-ac046dfd95fd None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:34.213286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827d72e-aa41-4c80-a044-ac046dfd95fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:34.239357 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-6827d72e-aa41-4c80-a044-ac046dfd95fd None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:34.240611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6827d72e-aa41-4c80-a044-ac046dfd95fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:34.252269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 764/1540] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:34.261158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a4a12-8413-49a7-80b9-2708d2f8fce8 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:34.261769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a4a12-8413-49a7-80b9-2708d2f8fce8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:34.261769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a4a12-8413-49a7-80b9-2708d2f8fce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:34.293342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6e7a4a12-8413-49a7-80b9-2708d2f8fce8 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:34.294565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6e7a4a12-8413-49a7-80b9-2708d2f8fce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:34.307398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 777/1541] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:34.311877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827d72e-aa41-4c80-a044-ac046dfd95fd None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:34.325517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827d72e-aa41-4c80-a044-ac046dfd95fd None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:34.332663 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec9ead-761a-4a66-85fa-3c5b5a5029a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:34.333279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec9ead-761a-4a66-85fa-3c5b5a5029a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:34.333562 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec9ead-761a-4a66-85fa-3c5b5a5029a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:34.333835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec9ead-761a-4a66-85fa-3c5b5a5029a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:34.334665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec9ead-761a-4a66-85fa-3c5b5a5029a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:29:34.334972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec9ead-761a-4a66-85fa-3c5b5a5029a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:34.336747 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec9ead-761a-4a66-85fa-3c5b5a5029a4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:34.355373 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 765/1542] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:29:34.364789 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a4a12-8413-49a7-80b9-2708d2f8fce8 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:34.378653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a4a12-8413-49a7-80b9-2708d2f8fce8 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:34.384025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ff75f-1bc2-46b1-b6d3-f686b8d4f440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:34.384660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ff75f-1bc2-46b1-b6d3-f686b8d4f440 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:34.384942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ff75f-1bc2-46b1-b6d3-f686b8d4f440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:34.385218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ff75f-1bc2-46b1-b6d3-f686b8d4f440 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:34.385872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ff75f-1bc2-46b1-b6d3-f686b8d4f440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:34.387896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ff75f-1bc2-46b1-b6d3-f686b8d4f440 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:34.454518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 778/1543] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:34 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:34.458966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec9ead-761a-4a66-85fa-3c5b5a5029a4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:34.471654 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec9ead-761a-4a66-85fa-3c5b5a5029a4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:34.560679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185e7eb-f523-443c-8409-2baed23493f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:34.561303 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185e7eb-f523-443c-8409-2baed23493f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:34.561585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185e7eb-f523-443c-8409-2baed23493f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:34.561857 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185e7eb-f523-443c-8409-2baed23493f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:34.562526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185e7eb-f523-443c-8409-2baed23493f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:34.564638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185e7eb-f523-443c-8409-2baed23493f9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:34.603721 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 766/1544] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:34 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:34.607619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ff75f-1bc2-46b1-b6d3-f686b8d4f440 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:34.632308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ff75f-1bc2-46b1-b6d3-f686b8d4f440 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:34.642122 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe0a00-d130-4d7f-a311-027cd13bbe2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:34.642768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0a00-d130-4d7f-a311-027cd13bbe2b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:34.643052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0a00-d130-4d7f-a311-027cd13bbe2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:34.649316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0a00-d130-4d7f-a311-027cd13bbe2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:34.650131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe0a00-d130-4d7f-a311-027cd13bbe2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:34.651821 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe0a00-d130-4d7f-a311-027cd13bbe2b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:34.669955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 779/1545] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:34.677644 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185e7eb-f523-443c-8409-2baed23493f9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:34.694341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185e7eb-f523-443c-8409-2baed23493f9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:34.702252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b1d95-6814-4614-8cc5-8acf89337c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:34.702902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b1d95-6814-4614-8cc5-8acf89337c21 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:34.703296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b1d95-6814-4614-8cc5-8acf89337c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:34.703578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b1d95-6814-4614-8cc5-8acf89337c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:34.704386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b1d95-6814-4614-8cc5-8acf89337c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:34.706568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b1d95-6814-4614-8cc5-8acf89337c21 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:34.730920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 767/1546] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:34.734717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe0a00-d130-4d7f-a311-027cd13bbe2b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:34.750680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe0a00-d130-4d7f-a311-027cd13bbe2b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:34.761020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570888c3-28da-43c8-b375-e308b948ef66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:34.761720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570888c3-28da-43c8-b375-e308b948ef66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:34.762011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570888c3-28da-43c8-b375-e308b948ef66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:34.762293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570888c3-28da-43c8-b375-e308b948ef66 admin admin] PATH_INFO: `/v3/projects/0a79d2a9724e49a9934d9ca0184b31fd/users/54b72a430d9d4d43954eb56d703079b4/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:34.791396 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570888c3-28da-43c8-b375-e308b948ef66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a79d2a9724e49a9934d9ca0184b31fd, user_id=54b72a430d9d4d43954eb56d703079b4, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:34.797259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570888c3-28da-43c8-b375-e308b948ef66 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:34.822582 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 780/1547] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:34 2022] PUT /identity/v3/projects/0a79d2a9724e49a9934d9ca0184b31fd/users/54b72a430d9d4d43954eb56d703079b4/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:34.828157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea268a-42e3-4477-b091-0307363b30fd None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:34.828568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea268a-42e3-4477-b091-0307363b30fd None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:34.829024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea268a-42e3-4477-b091-0307363b30fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:34.856579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3cea268a-42e3-4477-b091-0307363b30fd None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:34.857797 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3cea268a-42e3-4477-b091-0307363b30fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:34.918025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c0f3b-d01c-42a8-bd59-f6b00a1d8a3d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:34.918393 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c0f3b-d01c-42a8-bd59-f6b00a1d8a3d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:34.918571 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c0f3b-d01c-42a8-bd59-f6b00a1d8a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:34.949440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 768/1548] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:34.952438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2c1c0f3b-d01c-42a8-bd59-f6b00a1d8a3d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:34.954330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2c1c0f3b-d01c-42a8-bd59-f6b00a1d8a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:34.970683 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 781/1549] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:34.975919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea268a-42e3-4477-b091-0307363b30fd None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:34.979606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a72a50-9c2d-43b2-96d2-633163c0de53 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:34.979896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a72a50-9c2d-43b2-96d2-633163c0de53 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:34.980216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a72a50-9c2d-43b2-96d2-633163c0de53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.011853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-80a72a50-9c2d-43b2-96d2-633163c0de53 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:35.013012 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-80a72a50-9c2d-43b2-96d2-633163c0de53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:35.030818 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 782/1550] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:35.035350 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a72a50-9c2d-43b2-96d2-633163c0de53 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.050820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a72a50-9c2d-43b2-96d2-633163c0de53 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.067540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60523ac-ed82-4a64-abaa-38296a3fec7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.068198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60523ac-ed82-4a64-abaa-38296a3fec7b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.068633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60523ac-ed82-4a64-abaa-38296a3fec7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.069070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60523ac-ed82-4a64-abaa-38296a3fec7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.069854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60523ac-ed82-4a64-abaa-38296a3fec7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:29:35.070165 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60523ac-ed82-4a64-abaa-38296a3fec7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.071194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-a60523ac-ed82-4a64-abaa-38296a3fec7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:29:35.071194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 11 05:29:35.071194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: Sep 11 05:29:35.072117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60523ac-ed82-4a64-abaa-38296a3fec7b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.072923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea268a-42e3-4477-b091-0307363b30fd None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.082343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877c86e-507e-4e08-881c-fa0c41bc4b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.082980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877c86e-507e-4e08-881c-fa0c41bc4b56 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.083271 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877c86e-507e-4e08-881c-fa0c41bc4b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.083530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877c86e-507e-4e08-881c-fa0c41bc4b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.090347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 783/1551] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:29:35.091693 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b7831-b5a8-4a0c-b246-654ded1f38c5 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.091985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b7831-b5a8-4a0c-b246-654ded1f38c5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.096495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b7831-b5a8-4a0c-b246-654ded1f38c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.103907 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4877c86e-507e-4e08-881c-fa0c41bc4b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.105421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4877c86e-507e-4e08-881c-fa0c41bc4b56 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.126048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 769/1552] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:35.127327 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e7b9c-47ac-4622-8547-6b970bd5094d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.127614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e7b9c-47ac-4622-8547-6b970bd5094d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.127890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e7b9c-47ac-4622-8547-6b970bd5094d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.133547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6d8b7831-b5a8-4a0c-b246-654ded1f38c5 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:35.134871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6d8b7831-b5a8-4a0c-b246-654ded1f38c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:35.147476 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:35.148083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 784/1553] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:35.149392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385827f-146a-4ebb-8e5c-b4e140b0bae8 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.149788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385827f-146a-4ebb-8e5c-b4e140b0bae8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.149962 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385827f-146a-4ebb-8e5c-b4e140b0bae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.158686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-610e7b9c-47ac-4622-8547-6b970bd5094d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:35.159911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-610e7b9c-47ac-4622-8547-6b970bd5094d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:35.169121 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 770/1554] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:35.170008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e7b9c-47ac-4622-8547-6b970bd5094d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.173781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8385827f-146a-4ebb-8e5c-b4e140b0bae8 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:35.175055 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8385827f-146a-4ebb-8e5c-b4e140b0bae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:35.182967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e7b9c-47ac-4622-8547-6b970bd5094d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.183580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 785/1555] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:35.184976 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e131c-1366-4ad3-94df-ce06fc3c9483 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.185265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e131c-1366-4ad3-94df-ce06fc3c9483 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.185540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e131c-1366-4ad3-94df-ce06fc3c9483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.192176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107df4f-a955-48e3-928f-2667a9aa838b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.193153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107df4f-a955-48e3-928f-2667a9aa838b admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.193679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107df4f-a955-48e3-928f-2667a9aa838b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.194022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107df4f-a955-48e3-928f-2667a9aa838b admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.194606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107df4f-a955-48e3-928f-2667a9aa838b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.195680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-3107df4f-a955-48e3-928f-2667a9aa838b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:29:35.195680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Sep 11 05:29:35.195680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: Sep 11 05:29:35.196894 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107df4f-a955-48e3-928f-2667a9aa838b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.206437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-318e131c-1366-4ad3-94df-ce06fc3c9483 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:35.207867 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-318e131c-1366-4ad3-94df-ce06fc3c9483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:35.215895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 786/1556] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:35.217144 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f9d4b-dd50-4a3d-9505-4671c8183b02 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.217424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f9d4b-dd50-4a3d-9505-4671c8183b02 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.217698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f9d4b-dd50-4a3d-9505-4671c8183b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.238420 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e28f9d4b-dd50-4a3d-9505-4671c8183b02 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:35.238662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 771/1557] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/projects => generated 405 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:35.240137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3294af-7de6-4939-a05d-76a4c7cf488e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.240495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3294af-7de6-4939-a05d-76a4c7cf488e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.240864 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3294af-7de6-4939-a05d-76a4c7cf488e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.243182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e28f9d4b-dd50-4a3d-9505-4671c8183b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:35.263532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-1b3294af-7de6-4939-a05d-76a4c7cf488e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:35.264906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1b3294af-7de6-4939-a05d-76a4c7cf488e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:35.323303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 787/1558] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:35.323303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e82d6-5fb9-4bbc-b0cf-36782f95e5ef None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.323303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e82d6-5fb9-4bbc-b0cf-36782f95e5ef None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.323303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e82d6-5fb9-4bbc-b0cf-36782f95e5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.339040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 772/1559] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:35.340341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0823b3-8212-41c6-87c2-3f9a8e452712 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.340629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0823b3-8212-41c6-87c2-3f9a8e452712 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.341095 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0823b3-8212-41c6-87c2-3f9a8e452712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.341941 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3b3e82d6-5fb9-4bbc-b0cf-36782f95e5ef None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:35.343208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3b3e82d6-5fb9-4bbc-b0cf-36782f95e5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:35.361951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 788/1560] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:35.361951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15e9bf-183c-4a0c-8b6f-8c6fe0f16602 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.361951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15e9bf-183c-4a0c-8b6f-8c6fe0f16602 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.362313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15e9bf-183c-4a0c-8b6f-8c6fe0f16602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.366137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5a0823b3-8212-41c6-87c2-3f9a8e452712 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:35.367357 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5a0823b3-8212-41c6-87c2-3f9a8e452712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:35.376464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 773/1561] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:35.377432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c20a8-a1a3-4784-a68a-deac3f4ff1bd None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.377706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c20a8-a1a3-4784-a68a-deac3f4ff1bd None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.377982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c20a8-a1a3-4784-a68a-deac3f4ff1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.388252 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fc15e9bf-183c-4a0c-8b6f-8c6fe0f16602 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:35.389489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fc15e9bf-183c-4a0c-8b6f-8c6fe0f16602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:35.398183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 789/1562] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:35.399829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15e9bf-183c-4a0c-8b6f-8c6fe0f16602 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.400030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-054c20a8-a1a3-4784-a68a-deac3f4ff1bd None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:35.401222 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-054c20a8-a1a3-4784-a68a-deac3f4ff1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:35.414295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 774/1563] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:35.414295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e06b7f-bc7b-494d-b11f-ebd336482bc9 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.414295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e06b7f-bc7b-494d-b11f-ebd336482bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.414295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e06b7f-bc7b-494d-b11f-ebd336482bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.414736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15e9bf-183c-4a0c-8b6f-8c6fe0f16602 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.418556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39c906-94a4-4013-9d3e-6df31f1a4caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.419228 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39c906-94a4-4013-9d3e-6df31f1a4caa admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.419516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39c906-94a4-4013-9d3e-6df31f1a4caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.419788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39c906-94a4-4013-9d3e-6df31f1a4caa admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.420509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39c906-94a4-4013-9d3e-6df31f1a4caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.421538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-da39c906-94a4-4013-9d3e-6df31f1a4caa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:29:35.421538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 11 05:29:35.421538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: Sep 11 05:29:35.422948 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39c906-94a4-4013-9d3e-6df31f1a4caa admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.433509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-53e06b7f-bc7b-494d-b11f-ebd336482bc9 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:35.434921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-53e06b7f-bc7b-494d-b11f-ebd336482bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:35.446031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 775/1564] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:35.447743 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45fa29-9165-46bd-90e6-3b9e84846dfc None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.448116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45fa29-9165-46bd-90e6-3b9e84846dfc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.448449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45fa29-9165-46bd-90e6-3b9e84846dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.452372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 790/1565] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/users => generated 303 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:35.453808 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39c906-94a4-4013-9d3e-6df31f1a4caa admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.465063 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39c906-94a4-4013-9d3e-6df31f1a4caa admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.470961 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ce235-9d61-41f6-9aa4-99673ed407ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.471576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ce235-9d61-41f6-9aa4-99673ed407ea admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.471856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ce235-9d61-41f6-9aa4-99673ed407ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.472161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ce235-9d61-41f6-9aa4-99673ed407ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.472869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ce235-9d61-41f6-9aa4-99673ed407ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:29:35.473099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ac45fa29-9165-46bd-90e6-3b9e84846dfc None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:35.473227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ce235-9d61-41f6-9aa4-99673ed407ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.474437 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ac45fa29-9165-46bd-90e6-3b9e84846dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:35.474764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ce235-9d61-41f6-9aa4-99673ed407ea admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.483576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:35.484631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 776/1566] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:35.488441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45fa29-9165-46bd-90e6-3b9e84846dfc None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.495344 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 791/1567] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:29:35.496495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ce235-9d61-41f6-9aa4-99673ed407ea admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.506656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45fa29-9165-46bd-90e6-3b9e84846dfc None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.512201 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd6e8a-9c66-4499-8da4-625c6221cad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.512934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd6e8a-9c66-4499-8da4-625c6221cad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.513388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd6e8a-9c66-4499-8da4-625c6221cad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.513788 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd6e8a-9c66-4499-8da4-625c6221cad1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.514462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd6e8a-9c66-4499-8da4-625c6221cad1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:29:35.514654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ce235-9d61-41f6-9aa4-99673ed407ea admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.514897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd6e8a-9c66-4499-8da4-625c6221cad1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.517107 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd6e8a-9c66-4499-8da4-625c6221cad1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.519955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf9f40-4510-45bc-9e36-71f762acc56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.520590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf9f40-4510-45bc-9e36-71f762acc56a admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.520997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf9f40-4510-45bc-9e36-71f762acc56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.521280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf9f40-4510-45bc-9e36-71f762acc56a admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.521956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf9f40-4510-45bc-9e36-71f762acc56a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:29:35.522290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf9f40-4510-45bc-9e36-71f762acc56a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.523920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf9f40-4510-45bc-9e36-71f762acc56a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.541948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 777/1568] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:29:35.541948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd6e8a-9c66-4499-8da4-625c6221cad1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.542248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 792/1569] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:29:35.542248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf9f40-4510-45bc-9e36-71f762acc56a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.550623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd6e8a-9c66-4499-8da4-625c6221cad1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.554206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf9f40-4510-45bc-9e36-71f762acc56a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.558687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924f3e8-7260-4b95-a0f5-79df4c19890e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.559880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924f3e8-7260-4b95-a0f5-79df4c19890e admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.560432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924f3e8-7260-4b95-a0f5-79df4c19890e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.560965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924f3e8-7260-4b95-a0f5-79df4c19890e admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.562150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924f3e8-7260-4b95-a0f5-79df4c19890e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:29:35.562402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e9425-1f74-4626-85d2-f990027e7755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.563247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924f3e8-7260-4b95-a0f5-79df4c19890e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.564195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e9425-1f74-4626-85d2-f990027e7755 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.564638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e9425-1f74-4626-85d2-f990027e7755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.565081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e9425-1f74-4626-85d2-f990027e7755 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.565926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e9425-1f74-4626-85d2-f990027e7755 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:29:35.566148 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924f3e8-7260-4b95-a0f5-79df4c19890e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.566563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e9425-1f74-4626-85d2-f990027e7755 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.568412 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e9425-1f74-4626-85d2-f990027e7755 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.581486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 778/1570] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:29:35.584859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 793/1571] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:29:35.586176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924f3e8-7260-4b95-a0f5-79df4c19890e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.586393 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e9425-1f74-4626-85d2-f990027e7755 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.602531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e9425-1f74-4626-85d2-f990027e7755 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.607389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924f3e8-7260-4b95-a0f5-79df4c19890e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.614544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085daf95-a6f1-4c85-8ee7-6884085edb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.615127 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47bbd4-b1aa-4b61-8a7e-220bcd859108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.615836 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47bbd4-b1aa-4b61-8a7e-220bcd859108 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.616211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47bbd4-b1aa-4b61-8a7e-220bcd859108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.616465 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085daf95-a6f1-4c85-8ee7-6884085edb59 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.616637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47bbd4-b1aa-4b61-8a7e-220bcd859108 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.617105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085daf95-a6f1-4c85-8ee7-6884085edb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.617379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085daf95-a6f1-4c85-8ee7-6884085edb59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.618121 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085daf95-a6f1-4c85-8ee7-6884085edb59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:29:35.618430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085daf95-a6f1-4c85-8ee7-6884085edb59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.618528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47bbd4-b1aa-4b61-8a7e-220bcd859108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.620082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085daf95-a6f1-4c85-8ee7-6884085edb59 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.620215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47bbd4-b1aa-4b61-8a7e-220bcd859108 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.632110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 779/1572] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:35.633325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47bbd4-b1aa-4b61-8a7e-220bcd859108 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.635394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 794/1573] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:29:35.635394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085daf95-a6f1-4c85-8ee7-6884085edb59 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.647880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085daf95-a6f1-4c85-8ee7-6884085edb59 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.650884 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47bbd4-b1aa-4b61-8a7e-220bcd859108 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.653110 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882a553-18b4-4c21-8209-4d1b034e2afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.653892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882a553-18b4-4c21-8209-4d1b034e2afb admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.654269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882a553-18b4-4c21-8209-4d1b034e2afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.654704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882a553-18b4-4c21-8209-4d1b034e2afb admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.655453 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882a553-18b4-4c21-8209-4d1b034e2afb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.657017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e612f1-b4f5-4115-8ffe-6b8c0b9b5e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.657749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e612f1-b4f5-4115-8ffe-6b8c0b9b5e7d admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.658293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e612f1-b4f5-4115-8ffe-6b8c0b9b5e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.658388 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882a553-18b4-4c21-8209-4d1b034e2afb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.658477 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e612f1-b4f5-4115-8ffe-6b8c0b9b5e7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.659198 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e612f1-b4f5-4115-8ffe-6b8c0b9b5e7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.661122 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e612f1-b4f5-4115-8ffe-6b8c0b9b5e7d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.697255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 795/1574] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/projects => generated 391 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:35.697833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882a553-18b4-4c21-8209-4d1b034e2afb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.698496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 780/1575] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:35.699383 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e612f1-b4f5-4115-8ffe-6b8c0b9b5e7d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.760526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:35.779091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e612f1-b4f5-4115-8ffe-6b8c0b9b5e7d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.779920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882a553-18b4-4c21-8209-4d1b034e2afb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.785160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54d3df-cfeb-4b9e-871b-4c00997b0167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.785282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd1e0b-062e-4d73-8c26-b3cc767d93cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.785396 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54d3df-cfeb-4b9e-871b-4c00997b0167 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.785601 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54d3df-cfeb-4b9e-871b-4c00997b0167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.785865 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd1e0b-062e-4d73-8c26-b3cc767d93cf admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.786048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54d3df-cfeb-4b9e-871b-4c00997b0167 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.786130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd1e0b-062e-4d73-8c26-b3cc767d93cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.786378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd1e0b-062e-4d73-8c26-b3cc767d93cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.786770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f54d3df-cfeb-4b9e-871b-4c00997b0167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.787017 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd1e0b-062e-4d73-8c26-b3cc767d93cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.791984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f54d3df-cfeb-4b9e-871b-4c00997b0167 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.792181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd1e0b-062e-4d73-8c26-b3cc767d93cf admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.829432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 781/1576] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:35.830629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54d3df-cfeb-4b9e-871b-4c00997b0167 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.835214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 796/1577] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/projects => generated 369 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:35.836424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd1e0b-062e-4d73-8c26-b3cc767d93cf admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.903662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54d3df-cfeb-4b9e-871b-4c00997b0167 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.906522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd1e0b-062e-4d73-8c26-b3cc767d93cf admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.909530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ee437-c9cf-4363-829d-fa9b84c1f0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.910407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ee437-c9cf-4363-829d-fa9b84c1f0c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.910623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ee437-c9cf-4363-829d-fa9b84c1f0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.910922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ee437-c9cf-4363-829d-fa9b84c1f0c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.911798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4950a5-551a-4459-a2aa-0ada93be0e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.911926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ee437-c9cf-4363-829d-fa9b84c1f0c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.912053 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4950a5-551a-4459-a2aa-0ada93be0e6f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.912253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4950a5-551a-4459-a2aa-0ada93be0e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.912533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4950a5-551a-4459-a2aa-0ada93be0e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.913173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4950a5-551a-4459-a2aa-0ada93be0e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.913488 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ee437-c9cf-4363-829d-fa9b84c1f0c3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.914758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4950a5-551a-4459-a2aa-0ada93be0e6f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.929197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 797/1578] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:35.929197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4950a5-551a-4459-a2aa-0ada93be0e6f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.937087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4950a5-551a-4459-a2aa-0ada93be0e6f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:35.943409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b338dd5-47ba-49de-8652-0ea9452014df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:35.944178 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b338dd5-47ba-49de-8652-0ea9452014df admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:35.944464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b338dd5-47ba-49de-8652-0ea9452014df admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:35.944920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b338dd5-47ba-49de-8652-0ea9452014df admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:35.945498 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b338dd5-47ba-49de-8652-0ea9452014df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:35.947568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b338dd5-47ba-49de-8652-0ea9452014df admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:35.954378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 782/1579] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:35.955388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ee437-c9cf-4363-829d-fa9b84c1f0c3 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:35.972563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:35.989918 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 798/1580] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:35.991470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b338dd5-47ba-49de-8652-0ea9452014df admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.042461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ee437-c9cf-4363-829d-fa9b84c1f0c3 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.096763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b338dd5-47ba-49de-8652-0ea9452014df admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.103535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24849e6-e60a-492e-a7c8-7c9a0bee00c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.104423 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24849e6-e60a-492e-a7c8-7c9a0bee00c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.104857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24849e6-e60a-492e-a7c8-7c9a0bee00c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.105136 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24849e6-e60a-492e-a7c8-7c9a0bee00c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.106334 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24849e6-e60a-492e-a7c8-7c9a0bee00c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.109069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24849e6-e60a-492e-a7c8-7c9a0bee00c3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.127400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb04d6-3f82-49d1-b20d-477abafb64e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.128245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb04d6-3f82-49d1-b20d-477abafb64e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.128649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb04d6-3f82-49d1-b20d-477abafb64e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.129020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb04d6-3f82-49d1-b20d-477abafb64e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.129764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb04d6-3f82-49d1-b20d-477abafb64e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.132191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb04d6-3f82-49d1-b20d-477abafb64e7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.140069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 799/1581] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:36.141320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24849e6-e60a-492e-a7c8-7c9a0bee00c3 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.147326 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:36.158404 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24849e6-e60a-492e-a7c8-7c9a0bee00c3 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.163376 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 783/1582] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:35 2022] POST /identity/v3/users => generated 296 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:36.164312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb04d6-3f82-49d1-b20d-477abafb64e7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.165421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8e902-1145-4c84-abba-8003a405ef7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.166217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8e902-1145-4c84-abba-8003a405ef7e admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.166623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8e902-1145-4c84-abba-8003a405ef7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.167156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8e902-1145-4c84-abba-8003a405ef7e admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.167963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8e902-1145-4c84-abba-8003a405ef7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.170454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8e902-1145-4c84-abba-8003a405ef7e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.177101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb04d6-3f82-49d1-b20d-477abafb64e7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.183493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb828d6b-6484-4d4c-af08-b5d9c9281d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.184654 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb828d6b-6484-4d4c-af08-b5d9c9281d8e admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.185448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb828d6b-6484-4d4c-af08-b5d9c9281d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.185908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb828d6b-6484-4d4c-af08-b5d9c9281d8e admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.186807 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb828d6b-6484-4d4c-af08-b5d9c9281d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.189578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb828d6b-6484-4d4c-af08-b5d9c9281d8e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.198068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 800/1583] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:36 2022] POST /identity/v3/users => generated 294 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:36.203696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8e902-1145-4c84-abba-8003a405ef7e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.214297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 784/1584] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:36 2022] POST /identity/v3/users => generated 285 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:36.215499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb828d6b-6484-4d4c-af08-b5d9c9281d8e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.216628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8e902-1145-4c84-abba-8003a405ef7e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.224487 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9989b29-3e5e-4989-9df4-a18304b542fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.225272 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9989b29-3e5e-4989-9df4-a18304b542fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.225564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9989b29-3e5e-4989-9df4-a18304b542fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.225986 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9989b29-3e5e-4989-9df4-a18304b542fc admin admin] PATH_INFO: `/v3/projects/9394b17f9a8141fea20a24de7bf47d49/users/8eb77559a81745c8a897d1631663fa50/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.229056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb828d6b-6484-4d4c-af08-b5d9c9281d8e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.235281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f0d21-3d64-4300-be23-01e4d65d68bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.236742 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f0d21-3d64-4300-be23-01e4d65d68bb admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.237034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f0d21-3d64-4300-be23-01e4d65d68bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.237325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f0d21-3d64-4300-be23-01e4d65d68bb admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.237943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f0d21-3d64-4300-be23-01e4d65d68bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.239880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f0d21-3d64-4300-be23-01e4d65d68bb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.245802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9989b29-3e5e-4989-9df4-a18304b542fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9394b17f9a8141fea20a24de7bf47d49, user_id=8eb77559a81745c8a897d1631663fa50, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.249924 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9989b29-3e5e-4989-9df4-a18304b542fc admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.265107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 801/1585] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:36 2022] PUT /identity/v3/projects/9394b17f9a8141fea20a24de7bf47d49/users/8eb77559a81745c8a897d1631663fa50/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:36.266382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9989b29-3e5e-4989-9df4-a18304b542fc admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.269912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 785/1586] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:36 2022] POST /identity/v3/users => generated 289 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:36.270928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f0d21-3d64-4300-be23-01e4d65d68bb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.289404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f0d21-3d64-4300-be23-01e4d65d68bb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.357063 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9989b29-3e5e-4989-9df4-a18304b542fc admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.362869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebbae8-19f0-4eba-b1df-21d229314b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.363492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebbae8-19f0-4eba-b1df-21d229314b1c admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.363773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebbae8-19f0-4eba-b1df-21d229314b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.364254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebbae8-19f0-4eba-b1df-21d229314b1c admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.365227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ebbae8-19f0-4eba-b1df-21d229314b1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.367483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ebbae8-19f0-4eba-b1df-21d229314b1c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.375250 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26642e37-c694-4a74-a4e5-ff3787a4fdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.376166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26642e37-c694-4a74-a4e5-ff3787a4fdcd admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.376554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26642e37-c694-4a74-a4e5-ff3787a4fdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.376893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26642e37-c694-4a74-a4e5-ff3787a4fdcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.377592 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26642e37-c694-4a74-a4e5-ff3787a4fdcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.379275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26642e37-c694-4a74-a4e5-ff3787a4fdcd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.393156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 802/1587] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:36 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:36.394037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebbae8-19f0-4eba-b1df-21d229314b1c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.399196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 786/1588] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:36.400556 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26642e37-c694-4a74-a4e5-ff3787a4fdcd admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.410094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebbae8-19f0-4eba-b1df-21d229314b1c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.413684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26642e37-c694-4a74-a4e5-ff3787a4fdcd admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.416935 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83223e85-f434-414b-aade-ae451cc5d139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.417647 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83223e85-f434-414b-aade-ae451cc5d139 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.418018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83223e85-f434-414b-aade-ae451cc5d139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.418303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83223e85-f434-414b-aade-ae451cc5d139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.419388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a524302-6a26-4bd3-b5be-22ec8891a20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.419555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83223e85-f434-414b-aade-ae451cc5d139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.420078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a524302-6a26-4bd3-b5be-22ec8891a20e admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.420453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a524302-6a26-4bd3-b5be-22ec8891a20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.420789 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a524302-6a26-4bd3-b5be-22ec8891a20e admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.421498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a524302-6a26-4bd3-b5be-22ec8891a20e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.422362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83223e85-f434-414b-aade-ae451cc5d139 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.423699 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a524302-6a26-4bd3-b5be-22ec8891a20e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.435513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 787/1589] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:36.436160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 803/1590] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:36.437580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a1ed8-2649-4585-abdf-f7810f407a7c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.437928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a1ed8-2649-4585-abdf-f7810f407a7c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.438293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a1ed8-2649-4585-abdf-f7810f407a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.440331 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a524302-6a26-4bd3-b5be-22ec8891a20e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.454871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a524302-6a26-4bd3-b5be-22ec8891a20e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.467982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697d580-abd0-45c5-b26b-2109469cf775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.469342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697d580-abd0-45c5-b26b-2109469cf775 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.469643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697d580-abd0-45c5-b26b-2109469cf775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.470099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697d580-abd0-45c5-b26b-2109469cf775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.470755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697d580-abd0-45c5-b26b-2109469cf775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.472625 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697d580-abd0-45c5-b26b-2109469cf775 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.481273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6c7a1ed8-2649-4585-abdf-f7810f407a7c None None] MFA Rules not processed for user `8eb77559a81745c8a897d1631663fa50`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:36.483168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6c7a1ed8-2649-4585-abdf-f7810f407a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:36.486186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 788/1591] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:36.487307 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697d580-abd0-45c5-b26b-2109469cf775 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.497032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:36.497910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697d580-abd0-45c5-b26b-2109469cf775 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.509345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014da8e-d72c-48e2-932b-0c9663578f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.510576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014da8e-d72c-48e2-932b-0c9663578f5d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.511062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014da8e-d72c-48e2-932b-0c9663578f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.511515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014da8e-d72c-48e2-932b-0c9663578f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.512650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014da8e-d72c-48e2-932b-0c9663578f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.514425 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014da8e-d72c-48e2-932b-0c9663578f5d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.529370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 789/1592] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:36.530607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014da8e-d72c-48e2-932b-0c9663578f5d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.542840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014da8e-d72c-48e2-932b-0c9663578f5d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.548274 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95690fad-5c42-48b8-9db6-d9ec934d2115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.548892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95690fad-5c42-48b8-9db6-d9ec934d2115 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.549226 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95690fad-5c42-48b8-9db6-d9ec934d2115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.549510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95690fad-5c42-48b8-9db6-d9ec934d2115 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.550461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95690fad-5c42-48b8-9db6-d9ec934d2115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.553329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95690fad-5c42-48b8-9db6-d9ec934d2115 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.568377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 790/1593] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:36.568377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95690fad-5c42-48b8-9db6-d9ec934d2115 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.580245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95690fad-5c42-48b8-9db6-d9ec934d2115 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.584846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 804/1594] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:36 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:36.587786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a1ed8-2649-4585-abdf-f7810f407a7c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.589715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d6ab1-53d4-40de-a9b2-f82692d3b91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.590428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d6ab1-53d4-40de-a9b2-f82692d3b91d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.590765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d6ab1-53d4-40de-a9b2-f82692d3b91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.591926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d6ab1-53d4-40de-a9b2-f82692d3b91d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.592709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d6ab1-53d4-40de-a9b2-f82692d3b91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.595322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d6ab1-53d4-40de-a9b2-f82692d3b91d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.603056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a1ed8-2649-4585-abdf-f7810f407a7c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.610691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed0666-a9d9-4868-a4ac-cff68b7b1af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.611795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed0666-a9d9-4868-a4ac-cff68b7b1af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.612371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed0666-a9d9-4868-a4ac-cff68b7b1af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.612920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed0666-a9d9-4868-a4ac-cff68b7b1af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.613952 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed0666-a9d9-4868-a4ac-cff68b7b1af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.614782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 791/1595] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:36.616659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed0666-a9d9-4868-a4ac-cff68b7b1af6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.616781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d6ab1-53d4-40de-a9b2-f82692d3b91d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.628990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d6ab1-53d4-40de-a9b2-f82692d3b91d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.629855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 805/1596] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:36.630646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed0666-a9d9-4868-a4ac-cff68b7b1af6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.639537 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b3d29-9caf-435b-91e2-8a7acc8d3c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.640318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b3d29-9caf-435b-91e2-8a7acc8d3c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.641280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b3d29-9caf-435b-91e2-8a7acc8d3c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.641540 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b3d29-9caf-435b-91e2-8a7acc8d3c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.642309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b3d29-9caf-435b-91e2-8a7acc8d3c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.643983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b3d29-9caf-435b-91e2-8a7acc8d3c4c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.644454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed0666-a9d9-4868-a4ac-cff68b7b1af6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.652769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897eefef-1b49-4e74-914c-cda261b5c5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.653638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eefef-1b49-4e74-914c-cda261b5c5eb admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.654087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eefef-1b49-4e74-914c-cda261b5c5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.655210 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eefef-1b49-4e74-914c-cda261b5c5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.655958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897eefef-1b49-4e74-914c-cda261b5c5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.657446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 792/1597] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:36.658559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b3d29-9caf-435b-91e2-8a7acc8d3c4c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.663512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897eefef-1b49-4e74-914c-cda261b5c5eb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.669753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b3d29-9caf-435b-91e2-8a7acc8d3c4c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.676321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b45ef-be34-4868-bc3e-eac05d416f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.677150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b45ef-be34-4868-bc3e-eac05d416f33 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.677349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b45ef-be34-4868-bc3e-eac05d416f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.677625 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b45ef-be34-4868-bc3e-eac05d416f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.678290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b45ef-be34-4868-bc3e-eac05d416f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.679041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 806/1598] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:36.679941 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897eefef-1b49-4e74-914c-cda261b5c5eb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.683575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b45ef-be34-4868-bc3e-eac05d416f33 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.692570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897eefef-1b49-4e74-914c-cda261b5c5eb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.696231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 793/1599] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:36.697134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b45ef-be34-4868-bc3e-eac05d416f33 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.701092 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf0fa1-dfbf-4b71-922c-9845ce97493e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.702097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf0fa1-dfbf-4b71-922c-9845ce97493e admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.702693 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf0fa1-dfbf-4b71-922c-9845ce97493e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.704126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf0fa1-dfbf-4b71-922c-9845ce97493e admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.705075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf0fa1-dfbf-4b71-922c-9845ce97493e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.708295 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf0fa1-dfbf-4b71-922c-9845ce97493e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.710384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b45ef-be34-4868-bc3e-eac05d416f33 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.718981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adf10c-35a6-4f65-b53c-9859f4cd7ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.719979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adf10c-35a6-4f65-b53c-9859f4cd7ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.720279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adf10c-35a6-4f65-b53c-9859f4cd7ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.721241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adf10c-35a6-4f65-b53c-9859f4cd7ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.721890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82adf10c-35a6-4f65-b53c-9859f4cd7ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.724137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82adf10c-35a6-4f65-b53c-9859f4cd7ea9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.731059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 807/1600] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:36.732024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf0fa1-dfbf-4b71-922c-9845ce97493e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.746830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf0fa1-dfbf-4b71-922c-9845ce97493e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.754022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb56133-8222-42b4-8976-e6bf98455a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.754738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb56133-8222-42b4-8976-e6bf98455a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.755048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb56133-8222-42b4-8976-e6bf98455a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.755306 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb56133-8222-42b4-8976-e6bf98455a0a admin admin] PATH_INFO: `/v3/projects/431b72dd50e44a53a597b28404ace63c/users/065c988c57fa47e9b58af2914ff6306d/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.773681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 794/1601] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:36 2022] POST /identity/v3/projects => generated 405 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:36.775384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adf10c-35a6-4f65-b53c-9859f4cd7ea9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.777483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb56133-8222-42b4-8976-e6bf98455a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=431b72dd50e44a53a597b28404ace63c, user_id=065c988c57fa47e9b58af2914ff6306d, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.780333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb56133-8222-42b4-8976-e6bf98455a0a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.800771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 808/1602] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:36 2022] PUT /identity/v3/projects/431b72dd50e44a53a597b28404ace63c/users/065c988c57fa47e9b58af2914ff6306d/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:36.801408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb56133-8222-42b4-8976-e6bf98455a0a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.820970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:36.899122 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adf10c-35a6-4f65-b53c-9859f4cd7ea9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.905768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d36cc3-8d19-4a4a-b7c9-e21b4832bee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.907275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d36cc3-8d19-4a4a-b7c9-e21b4832bee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.907655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d36cc3-8d19-4a4a-b7c9-e21b4832bee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.908024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d36cc3-8d19-4a4a-b7c9-e21b4832bee9 admin admin] PATH_INFO: `/v3/projects/60788ee7f0ab439087f63db271b65ef4/users/f4c978e96d3248d59120c287ecc75e87/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.912634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb56133-8222-42b4-8976-e6bf98455a0a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:36.925031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d36cc3-8d19-4a4a-b7c9-e21b4832bee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60788ee7f0ab439087f63db271b65ef4, user_id=f4c978e96d3248d59120c287ecc75e87, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.926163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9213-e1aa-4c47-9f0b-0258abc2af7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:36.926301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-51d36cc3-8d19-4a4a-b7c9-e21b4832bee9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:29:36.926301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Sep 11 05:29:36.926301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: Sep 11 05:29:36.926895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9213-e1aa-4c47-9f0b-0258abc2af7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:36.927224 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9213-e1aa-4c47-9f0b-0258abc2af7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:36.927650 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d9213-e1aa-4c47-9f0b-0258abc2af7a admin admin] PATH_INFO: `/v3/projects/37c662b75f1d46619c3993132d413549/users/0d1dbe40cb994491bafc6ce1f947f4f6/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:36.929230 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d36cc3-8d19-4a4a-b7c9-e21b4832bee9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.949444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 795/1603] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:36 2022] PUT /identity/v3/projects/60788ee7f0ab439087f63db271b65ef4/users/f4c978e96d3248d59120c287ecc75e87/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:36.950590 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d36cc3-8d19-4a4a-b7c9-e21b4832bee9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:36.957735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d9213-e1aa-4c47-9f0b-0258abc2af7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c662b75f1d46619c3993132d413549, user_id=0d1dbe40cb994491bafc6ce1f947f4f6, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:36.960439 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d9213-e1aa-4c47-9f0b-0258abc2af7a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:36.979796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 809/1604] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:36 2022] PUT /identity/v3/projects/37c662b75f1d46619c3993132d413549/users/0d1dbe40cb994491bafc6ce1f947f4f6/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:36.980756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9213-e1aa-4c47-9f0b-0258abc2af7a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:37.044712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:37.055515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d36cc3-8d19-4a4a-b7c9-e21b4832bee9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:37.075184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d9213-e1aa-4c47-9f0b-0258abc2af7a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:37.081820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a0bc6-0ff6-4683-90ea-bc2fdbb276d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:37.082807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a0bc6-0ff6-4683-90ea-bc2fdbb276d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.083029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a0bc6-0ff6-4683-90ea-bc2fdbb276d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.083402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a0bc6-0ff6-4683-90ea-bc2fdbb276d0 admin admin] PATH_INFO: `/v3/projects/e65940f72e054523aa100c10e3824c37/users/116ea262cc704290b7c787a0f39949b5/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.110265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a0bc6-0ff6-4683-90ea-bc2fdbb276d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e65940f72e054523aa100c10e3824c37, user_id=116ea262cc704290b7c787a0f39949b5, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:37.113403 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a0bc6-0ff6-4683-90ea-bc2fdbb276d0 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:37.128670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 810/1605] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:36 2022] PUT /identity/v3/projects/e65940f72e054523aa100c10e3824c37/users/116ea262cc704290b7c787a0f39949b5/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:37.129582 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a0bc6-0ff6-4683-90ea-bc2fdbb276d0 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:37.147746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff660f6-ac6f-4326-9734-489495db5326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:37.148465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff660f6-ac6f-4326-9734-489495db5326 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.148742 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff660f6-ac6f-4326-9734-489495db5326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.149099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff660f6-ac6f-4326-9734-489495db5326 admin admin] PATH_INFO: `/v3/projects/9c6ae8fc112c4e8a80d458ee4fadb479/users/5cb7b8d14d2747bfabf54a386f92c4e9/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.232083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff660f6-ac6f-4326-9734-489495db5326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c6ae8fc112c4e8a80d458ee4fadb479, user_id=5cb7b8d14d2747bfabf54a386f92c4e9, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:37.232083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff660f6-ac6f-4326-9734-489495db5326 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:37.232634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a0bc6-0ff6-4683-90ea-bc2fdbb276d0 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:37.232839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 796/1606] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:36 2022] PUT /identity/v3/projects/9c6ae8fc112c4e8a80d458ee4fadb479/users/5cb7b8d14d2747bfabf54a386f92c4e9/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:37.232839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff660f6-ac6f-4326-9734-489495db5326 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:37.292328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:37.323299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807064e9-a75f-4890-8a56-c04e1da1831d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:37.323981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807064e9-a75f-4890-8a56-c04e1da1831d admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.325500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807064e9-a75f-4890-8a56-c04e1da1831d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.325773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807064e9-a75f-4890-8a56-c04e1da1831d admin admin] PATH_INFO: `/v3/projects/d47e270753bc4fe6ac5ba1f2b0d5d2af/users/32df696da09746748786b94aa8b6721d/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.339667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff660f6-ac6f-4326-9734-489495db5326 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:37.345565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff3505-b58a-470c-b8f0-1bb5b27ac011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:37.346189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff3505-b58a-470c-b8f0-1bb5b27ac011 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.346466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff3505-b58a-470c-b8f0-1bb5b27ac011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.346735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff3505-b58a-470c-b8f0-1bb5b27ac011 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.347010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807064e9-a75f-4890-8a56-c04e1da1831d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d47e270753bc4fe6ac5ba1f2b0d5d2af, user_id=32df696da09746748786b94aa8b6721d, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:37.347436 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff3505-b58a-470c-b8f0-1bb5b27ac011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:37.349513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff3505-b58a-470c-b8f0-1bb5b27ac011 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:37.351327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807064e9-a75f-4890-8a56-c04e1da1831d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:37.369981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 811/1607] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:37 2022] PUT /identity/v3/projects/d47e270753bc4fe6ac5ba1f2b0d5d2af/users/32df696da09746748786b94aa8b6721d/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:37.377073 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0daadd-77e1-4be4-a00b-8a57340815c2 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.377400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0daadd-77e1-4be4-a00b-8a57340815c2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.377707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0daadd-77e1-4be4-a00b-8a57340815c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.384683 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 797/1608] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:37 2022] POST /identity/v3/users => generated 309 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:37.386255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c011f94-4955-47c3-af19-3af72e2885d6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.386784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c011f94-4955-47c3-af19-3af72e2885d6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.387138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c011f94-4955-47c3-af19-3af72e2885d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.401320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2d0daadd-77e1-4be4-a00b-8a57340815c2 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:37.402543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2d0daadd-77e1-4be4-a00b-8a57340815c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:37.410935 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5c011f94-4955-47c3-af19-3af72e2885d6 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:37.413973 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5c011f94-4955-47c3-af19-3af72e2885d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:37.456399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:37.480158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 812/1609] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:37.481557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e28a3-1673-419a-b600-f2e4abbf0e5d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.481843 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e28a3-1673-419a-b600-f2e4abbf0e5d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.482117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e28a3-1673-419a-b600-f2e4abbf0e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.494790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 798/1610] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:37.496086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5652-c45b-4ee0-bf1f-e9371bd82e00 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.496952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5652-c45b-4ee0-bf1f-e9371bd82e00 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.497255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5652-c45b-4ee0-bf1f-e9371bd82e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.505517 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-898e28a3-1673-419a-b600-f2e4abbf0e5d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:37.506795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-898e28a3-1673-419a-b600-f2e4abbf0e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:37.517366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 813/1611] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:37.519324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa65bb3-5edc-4ad6-8075-12e8f3444c74 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.519604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa65bb3-5edc-4ad6-8075-12e8f3444c74 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.519900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa65bb3-5edc-4ad6-8075-12e8f3444c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.541050 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-dd0f5652-c45b-4ee0-bf1f-e9371bd82e00 None None] MFA Rules not processed for user `116ea262cc704290b7c787a0f39949b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:37.542314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-dd0f5652-c45b-4ee0-bf1f-e9371bd82e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:37.553081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-efa65bb3-5edc-4ad6-8075-12e8f3444c74 None None] MFA Rules not processed for user `5cb7b8d14d2747bfabf54a386f92c4e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:37.554491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-efa65bb3-5edc-4ad6-8075-12e8f3444c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:37.624159 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 799/1612] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:37.626979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4b10e-b506-45d4-9c49-0ed6dc931454 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.627264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4b10e-b506-45d4-9c49-0ed6dc931454 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.627536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4b10e-b506-45d4-9c49-0ed6dc931454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.632383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 814/1613] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:37.633506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa65bb3-5edc-4ad6-8075-12e8f3444c74 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:37.648952 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa65bb3-5edc-4ad6-8075-12e8f3444c74 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:37.650387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d3a4b10e-b506-45d4-9c49-0ed6dc931454 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:37.651601 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d3a4b10e-b506-45d4-9c49-0ed6dc931454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:37.654209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100cb85-63fa-4d11-96cd-48b9bc7717cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:37.654845 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100cb85-63fa-4d11-96cd-48b9bc7717cb admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.655126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100cb85-63fa-4d11-96cd-48b9bc7717cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.655400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100cb85-63fa-4d11-96cd-48b9bc7717cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.656103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100cb85-63fa-4d11-96cd-48b9bc7717cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:37.658604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100cb85-63fa-4d11-96cd-48b9bc7717cb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:37.664216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 800/1614] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:37.664216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4b10e-b506-45d4-9c49-0ed6dc931454 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:37.673383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 815/1615] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:37.673383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100cb85-63fa-4d11-96cd-48b9bc7717cb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:37.675187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f100cb85-63fa-4d11-96cd-48b9bc7717cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:37.727504 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:37.740926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4b10e-b506-45d4-9c49-0ed6dc931454 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:37.747057 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cdbe1-bae6-4ac3-b624-a5c2b4a54371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:37.747596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cdbe1-bae6-4ac3-b624-a5c2b4a54371 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.747883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cdbe1-bae6-4ac3-b624-a5c2b4a54371 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.748320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cdbe1-bae6-4ac3-b624-a5c2b4a54371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.749465 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100cb85-63fa-4d11-96cd-48b9bc7717cb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:37.754094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff2c1c-a83f-4a9e-babd-c15c66626be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:37.754716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff2c1c-a83f-4a9e-babd-c15c66626be6 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.755000 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff2c1c-a83f-4a9e-babd-c15c66626be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.755736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff2c1c-a83f-4a9e-babd-c15c66626be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.766491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff2c1c-a83f-4a9e-babd-c15c66626be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:37.768156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff2c1c-a83f-4a9e-babd-c15c66626be6 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:37.769799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5cdbe1-bae6-4ac3-b624-a5c2b4a54371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:37.771522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5cdbe1-bae6-4ac3-b624-a5c2b4a54371 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:37.780155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 816/1616] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:37.781912 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff2c1c-a83f-4a9e-babd-c15c66626be6 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:37.785445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 801/1617] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:37.786480 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cdbe1-bae6-4ac3-b624-a5c2b4a54371 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:37.791730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff2c1c-a83f-4a9e-babd-c15c66626be6 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:37.796561 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1ab97-9c8a-40b4-88bb-c351c8576aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:37.798132 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1ab97-9c8a-40b4-88bb-c351c8576aee service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.798547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1ab97-9c8a-40b4-88bb-c351c8576aee service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.798938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1ab97-9c8a-40b4-88bb-c351c8576aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.799106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cdbe1-bae6-4ac3-b624-a5c2b4a54371 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:37.805471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa59328-6698-49b5-a17e-f485c6c399c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:37.806418 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa59328-6698-49b5-a17e-f485c6c399c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.806815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa59328-6698-49b5-a17e-f485c6c399c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.807223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa59328-6698-49b5-a17e-f485c6c399c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.807968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa59328-6698-49b5-a17e-f485c6c399c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:37.810678 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa59328-6698-49b5-a17e-f485c6c399c6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:37.813727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1ab97-9c8a-40b4-88bb-c351c8576aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:37.815311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1ab97-9c8a-40b4-88bb-c351c8576aee service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:37.836952 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 817/1618] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:37.838379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1ab97-9c8a-40b4-88bb-c351c8576aee service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:37.851421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 802/1619] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:37 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:37.853583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa59328-6698-49b5-a17e-f485c6c399c6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:37.856071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1ab97-9c8a-40b4-88bb-c351c8576aee service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:37.939397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:37.953850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa59328-6698-49b5-a17e-f485c6c399c6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:37.959080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f278b43-df46-49ef-8ae5-429c32d312ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:37.959413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62af0-d090-4a21-ac6c-7274c0eb9fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:37.959710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f278b43-df46-49ef-8ae5-429c32d312ba admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.960038 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f278b43-df46-49ef-8ae5-429c32d312ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.960505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62af0-d090-4a21-ac6c-7274c0eb9fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:37.960939 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62af0-d090-4a21-ac6c-7274c0eb9fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:37.961460 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b62af0-d090-4a21-ac6c-7274c0eb9fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.967694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f278b43-df46-49ef-8ae5-429c32d312ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:37.968766 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f278b43-df46-49ef-8ae5-429c32d312ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:37.983005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f278b43-df46-49ef-8ae5-429c32d312ba admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:38.005182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b62af0-d090-4a21-ac6c-7274c0eb9fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:38.011508 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b62af0-d090-4a21-ac6c-7274c0eb9fb2 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:38.029946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 818/1620] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:37 2022] POST /identity/v3/projects => generated 373 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:38.032783 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f278b43-df46-49ef-8ae5-429c32d312ba admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:38.037306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 803/1621] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:38.038321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62af0-d090-4a21-ac6c-7274c0eb9fb2 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:38.052874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b62af0-d090-4a21-ac6c-7274c0eb9fb2 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:38.138558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebee19-809f-419b-bdcf-82c578b0fc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:38.139705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebee19-809f-419b-bdcf-82c578b0fc12 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:38.140126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebee19-809f-419b-bdcf-82c578b0fc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:38.141676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebee19-809f-419b-bdcf-82c578b0fc12 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:38.142709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bebee19-809f-419b-bdcf-82c578b0fc12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:38.146421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bebee19-809f-419b-bdcf-82c578b0fc12 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:38.191026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 804/1622] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:38 2022] POST /identity/v3/users => generated 295 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:38.194454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebee19-809f-419b-bdcf-82c578b0fc12 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:38.199759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f278b43-df46-49ef-8ae5-429c32d312ba admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:38.213931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a9d5d-034a-409e-92c2-01c6318cb57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:38.216771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a9d5d-034a-409e-92c2-01c6318cb57c admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:38.217118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a9d5d-034a-409e-92c2-01c6318cb57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:38.217467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a9d5d-034a-409e-92c2-01c6318cb57c admin admin] PATH_INFO: `/v3/projects/db63b71df8c547ec8afb2c623f4673d1/users/34409e507bd24d5e93bfd4947b74f591/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:38.218174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebee19-809f-419b-bdcf-82c578b0fc12 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:38.229780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8703e-71a4-4acc-9200-d393d58941e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:38.231772 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8703e-71a4-4acc-9200-d393d58941e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:38.231924 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8703e-71a4-4acc-9200-d393d58941e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:38.232119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8703e-71a4-4acc-9200-d393d58941e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:38.233005 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8703e-71a4-4acc-9200-d393d58941e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:38.238903 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8703e-71a4-4acc-9200-d393d58941e3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:38.256866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a9d5d-034a-409e-92c2-01c6318cb57c admin admin] RBAC: Authorizing `identity:create_grant(project_id=db63b71df8c547ec8afb2c623f4673d1, user_id=34409e507bd24d5e93bfd4947b74f591, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:38.264189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a9d5d-034a-409e-92c2-01c6318cb57c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:38.300034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 805/1623] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:38 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:38.301000 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 819/1624] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:38 2022] PUT /identity/v3/projects/db63b71df8c547ec8afb2c623f4673d1/users/34409e507bd24d5e93bfd4947b74f591/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:38.301527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8703e-71a4-4acc-9200-d393d58941e3 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:38.306321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a9d5d-034a-409e-92c2-01c6318cb57c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:38.324256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8703e-71a4-4acc-9200-d393d58941e3 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:38.377966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:38.399614 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a9d5d-034a-409e-92c2-01c6318cb57c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:38.403189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392953b-1d4f-454c-b572-a3236ec2bb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:38.403914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392953b-1d4f-454c-b572-a3236ec2bb62 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:38.404692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392953b-1d4f-454c-b572-a3236ec2bb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:38.405044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392953b-1d4f-454c-b572-a3236ec2bb62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:38.405354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83539449-47c7-4511-8637-178d169a5b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:38.406639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83539449-47c7-4511-8637-178d169a5b02 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:38.407107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83539449-47c7-4511-8637-178d169a5b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:38.407750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83539449-47c7-4511-8637-178d169a5b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:38.408390 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392953b-1d4f-454c-b572-a3236ec2bb62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:38.408777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83539449-47c7-4511-8637-178d169a5b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:38.410321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392953b-1d4f-454c-b572-a3236ec2bb62 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:38.410530 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83539449-47c7-4511-8637-178d169a5b02 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:38.429499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 806/1626] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:38.429700 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 820/1626] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:38.429785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392953b-1d4f-454c-b572-a3236ec2bb62 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:38.429885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83539449-47c7-4511-8637-178d169a5b02 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:38.437101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392953b-1d4f-454c-b572-a3236ec2bb62 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:38.440120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83539449-47c7-4511-8637-178d169a5b02 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:38.445701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ccb5e-4006-4ed0-afd4-cc8b8201452f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:38.447456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb1028-bb44-47f1-bca5-90a515495a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:38.447711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ccb5e-4006-4ed0-afd4-cc8b8201452f admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:38.447818 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb1028-bb44-47f1-bca5-90a515495a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:38.447983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ccb5e-4006-4ed0-afd4-cc8b8201452f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:38.448278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb1028-bb44-47f1-bca5-90a515495a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:38.448567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb1028-bb44-47f1-bca5-90a515495a8c admin admin] PATH_INFO: `/v3/projects/db63b71df8c547ec8afb2c623f4673d1/users/34409e507bd24d5e93bfd4947b74f591/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:38.450259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ccb5e-4006-4ed0-afd4-cc8b8201452f admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:38.450610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb1028-bb44-47f1-bca5-90a515495a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=db63b71df8c547ec8afb2c623f4673d1, user_id=34409e507bd24d5e93bfd4947b74f591, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:38.450984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ccb5e-4006-4ed0-afd4-cc8b8201452f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:38.453755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb1028-bb44-47f1-bca5-90a515495a8c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:38.457794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ccb5e-4006-4ed0-afd4-cc8b8201452f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:38.475367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 821/1627] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:38 2022] PUT /identity/v3/projects/db63b71df8c547ec8afb2c623f4673d1/users/34409e507bd24d5e93bfd4947b74f591/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:38.475367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb1028-bb44-47f1-bca5-90a515495a8c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:38.494082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 807/1628] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:38.494082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b5009-d553-4def-ab0d-155d47916a45 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:38.494082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b5009-d553-4def-ab0d-155d47916a45 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:38.494082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b5009-d553-4def-ab0d-155d47916a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:38.546493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-1f3b5009-d553-4def-ab0d-155d47916a45 None None] MFA Rules not processed for user `34409e507bd24d5e93bfd4947b74f591`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:38.557132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1f3b5009-d553-4def-ab0d-155d47916a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:38.642574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:38.691720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb1028-bb44-47f1-bca5-90a515495a8c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:38.711125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4be50b-0a38-48ba-9d07-8a5ad20888e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:38.711856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4be50b-0a38-48ba-9d07-8a5ad20888e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:38.712199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4be50b-0a38-48ba-9d07-8a5ad20888e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:38.712505 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4be50b-0a38-48ba-9d07-8a5ad20888e5 admin admin] PATH_INFO: `/v3/projects/de09720fe1b44d60ba0043da6668239a/users/d46931fa823b40a4b1171ea57a57b121/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:38.754535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4be50b-0a38-48ba-9d07-8a5ad20888e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de09720fe1b44d60ba0043da6668239a, user_id=d46931fa823b40a4b1171ea57a57b121, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:38.760431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4be50b-0a38-48ba-9d07-8a5ad20888e5 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:38.785279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 808/1629] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:38 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:38.786516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b5009-d553-4def-ab0d-155d47916a45 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:38.799364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 822/1630] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:38 2022] PUT /identity/v3/projects/de09720fe1b44d60ba0043da6668239a/users/d46931fa823b40a4b1171ea57a57b121/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:38.804493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4be50b-0a38-48ba-9d07-8a5ad20888e5 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:38.814458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b5009-d553-4def-ab0d-155d47916a45 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:38.906015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4be50b-0a38-48ba-9d07-8a5ad20888e5 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:38.920838 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944de72-2432-456c-8a6d-b9ff4cf3974f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:38.921495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944de72-2432-456c-8a6d-b9ff4cf3974f service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:38.922336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944de72-2432-456c-8a6d-b9ff4cf3974f service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:38.922784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944de72-2432-456c-8a6d-b9ff4cf3974f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:38.968139 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da946f25-eea9-4408-8770-97bcdaf475cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:38.968716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da946f25-eea9-4408-8770-97bcdaf475cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:38.968906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da946f25-eea9-4408-8770-97bcdaf475cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:38.969254 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da946f25-eea9-4408-8770-97bcdaf475cf admin admin] PATH_INFO: `/v3/projects/7aff275667bc480a8df0cd1a0a0c1a3c/users/8eb4ccf85fca43c483b17173496cdf5c/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:39.004035 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da946f25-eea9-4408-8770-97bcdaf475cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aff275667bc480a8df0cd1a0a0c1a3c, user_id=8eb4ccf85fca43c483b17173496cdf5c, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:39.007203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da946f25-eea9-4408-8770-97bcdaf475cf admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:39.038320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 809/1631] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:38 2022] PUT /identity/v3/projects/7aff275667bc480a8df0cd1a0a0c1a3c/users/8eb4ccf85fca43c483b17173496cdf5c/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:39.039393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da946f25-eea9-4408-8770-97bcdaf475cf admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:39.048966 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944de72-2432-456c-8a6d-b9ff4cf3974f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:39.050443 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944de72-2432-456c-8a6d-b9ff4cf3974f service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:39.073718 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 823/1632] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:38 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:39.083355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944de72-2432-456c-8a6d-b9ff4cf3974f service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:39.113234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944de72-2432-456c-8a6d-b9ff4cf3974f service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:39.162726 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:39.268276 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da946f25-eea9-4408-8770-97bcdaf475cf admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:39.274456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e67647-506e-48f9-823c-7bff1a9bef48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:39.275095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e67647-506e-48f9-823c-7bff1a9bef48 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:39.275380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e67647-506e-48f9-823c-7bff1a9bef48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:39.275663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e67647-506e-48f9-823c-7bff1a9bef48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:39.276351 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e67647-506e-48f9-823c-7bff1a9bef48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:39.284517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0a2d2-e5f2-4df9-b9cb-4772b0bf7cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:39.285214 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0a2d2-e5f2-4df9-b9cb-4772b0bf7cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:39.285499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0a2d2-e5f2-4df9-b9cb-4772b0bf7cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:39.285765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0a2d2-e5f2-4df9-b9cb-4772b0bf7cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:39.286419 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0a2d2-e5f2-4df9-b9cb-4772b0bf7cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:39.286593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e67647-506e-48f9-823c-7bff1a9bef48 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:39.288846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0a2d2-e5f2-4df9-b9cb-4772b0bf7cd4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:39.306458 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 824/1633] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:39.316652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 810/1634] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:39.319147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e67647-506e-48f9-823c-7bff1a9bef48 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:39.320724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0a2d2-e5f2-4df9-b9cb-4772b0bf7cd4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:39.341358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e67647-506e-48f9-823c-7bff1a9bef48 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:39.345435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0a2d2-e5f2-4df9-b9cb-4772b0bf7cd4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:39.353820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722c153-f6bf-4ba1-bc12-b28ec9d7748f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:39.354776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722c153-f6bf-4ba1-bc12-b28ec9d7748f admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:39.355729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722c153-f6bf-4ba1-bc12-b28ec9d7748f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:39.356557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722c153-f6bf-4ba1-bc12-b28ec9d7748f admin admin] PATH_INFO: `/v3/projects/7aff275667bc480a8df0cd1a0a0c1a3c/users/8eb4ccf85fca43c483b17173496cdf5c/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:39.358906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e54d5-24ae-426c-b13b-b5f9bdcdab72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:39.361112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e54d5-24ae-426c-b13b-b5f9bdcdab72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:39.362071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c722c153-f6bf-4ba1-bc12-b28ec9d7748f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aff275667bc480a8df0cd1a0a0c1a3c, user_id=8eb4ccf85fca43c483b17173496cdf5c, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:39.362561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e54d5-24ae-426c-b13b-b5f9bdcdab72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:39.364503 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e54d5-24ae-426c-b13b-b5f9bdcdab72 admin admin] PATH_INFO: `/v3/projects/de09720fe1b44d60ba0043da6668239a/users/d46931fa823b40a4b1171ea57a57b121/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:39.368583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c722c153-f6bf-4ba1-bc12-b28ec9d7748f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:39.369552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e54d5-24ae-426c-b13b-b5f9bdcdab72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de09720fe1b44d60ba0043da6668239a, user_id=d46931fa823b40a4b1171ea57a57b121, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:39.380158 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e54d5-24ae-426c-b13b-b5f9bdcdab72 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:39.395733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:39.418131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 825/1635] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:39 2022] PUT /identity/v3/projects/7aff275667bc480a8df0cd1a0a0c1a3c/users/8eb4ccf85fca43c483b17173496cdf5c/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:39.420076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16397118-5c6c-4998-ba27-dd100712e7be None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:39.422527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 811/1636] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:39 2022] PUT /identity/v3/projects/de09720fe1b44d60ba0043da6668239a/users/d46931fa823b40a4b1171ea57a57b121/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:39.425446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16397118-5c6c-4998-ba27-dd100712e7be None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:39.425846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16397118-5c6c-4998-ba27-dd100712e7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:39.435089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8f4dd-d976-41cf-aaf2-6a0024363783 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:39.435089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8f4dd-d976-41cf-aaf2-6a0024363783 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:39.435089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8f4dd-d976-41cf-aaf2-6a0024363783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:39.497773 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-42f8f4dd-d976-41cf-aaf2-6a0024363783 None None] MFA Rules not processed for user `d46931fa823b40a4b1171ea57a57b121`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:39.499027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-42f8f4dd-d976-41cf-aaf2-6a0024363783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:39.500764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-16397118-5c6c-4998-ba27-dd100712e7be None None] MFA Rules not processed for user `8eb4ccf85fca43c483b17173496cdf5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:39.502081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-16397118-5c6c-4998-ba27-dd100712e7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:39.690547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 826/1637] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:39.690547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16397118-5c6c-4998-ba27-dd100712e7be None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:39.699268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 812/1638] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:39.711501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8f4dd-d976-41cf-aaf2-6a0024363783 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:39.879464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8f4dd-d976-41cf-aaf2-6a0024363783 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:39.892749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c60c8-b13a-433a-b845-688bbcdfb9c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:39.894189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c60c8-b13a-433a-b845-688bbcdfb9c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:39.895392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c60c8-b13a-433a-b845-688bbcdfb9c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:39.897554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c60c8-b13a-433a-b845-688bbcdfb9c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:39.900368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16397118-5c6c-4998-ba27-dd100712e7be None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:39.918479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61315dda-e0ac-4576-a733-6425031a879d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:39.919748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61315dda-e0ac-4576-a733-6425031a879d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:39.920359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61315dda-e0ac-4576-a733-6425031a879d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:39.920908 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61315dda-e0ac-4576-a733-6425031a879d admin admin] PATH_INFO: `/v3/users/8eb77559a81745c8a897d1631663fa50` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:39.922785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61315dda-e0ac-4576-a733-6425031a879d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb77559a81745c8a897d1631663fa50)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:39.929340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61315dda-e0ac-4576-a733-6425031a879d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:39.929625 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c60c8-b13a-433a-b845-688bbcdfb9c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:39.931105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c60c8-b13a-433a-b845-688bbcdfb9c8 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:39.979119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 813/1639] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:29:39.983876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c60c8-b13a-433a-b845-688bbcdfb9c8 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:40.020275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c60c8-b13a-433a-b845-688bbcdfb9c8 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:40.027931 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2bca86-e0e8-47b4-9f21-4f1f957508e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:40.029628 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bca86-e0e8-47b4-9f21-4f1f957508e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:40.029984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bca86-e0e8-47b4-9f21-4f1f957508e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:40.030381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bca86-e0e8-47b4-9f21-4f1f957508e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:40.056232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-61315dda-e0ac-4576-a733-6425031a879d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb77559a81745c8a897d1631663fa50'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:40.057397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-61315dda-e0ac-4576-a733-6425031a879d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb77559a81745c8a897d1631663fa50'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:40.073158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-61315dda-e0ac-4576-a733-6425031a879d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb77559a81745c8a897d1631663fa50'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:40.090569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-61315dda-e0ac-4576-a733-6425031a879d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb77559a81745c8a897d1631663fa50'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:40.114820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-61315dda-e0ac-4576-a733-6425031a879d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb77559a81745c8a897d1631663fa50'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:40.129682 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 827/1640] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:29:39 2022] DELETE /identity/v3/users/8eb77559a81745c8a897d1631663fa50 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:40.135304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c3ccf-3b62-4185-aa09-bff51998109c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:40.136382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c3ccf-3b62-4185-aa09-bff51998109c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:40.136868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c3ccf-3b62-4185-aa09-bff51998109c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:40.150588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:40.157540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:40.189013 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5e5c3ccf-3b62-4185-aa09-bff51998109c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:40.190373 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5e5c3ccf-3b62-4185-aa09-bff51998109c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:40.200473 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2bca86-e0e8-47b4-9f21-4f1f957508e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:40.203969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2bca86-e0e8-47b4-9f21-4f1f957508e1 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:40.236810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 814/1641] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:40.355719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 828/1642] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:40.424243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2bca86-e0e8-47b4-9f21-4f1f957508e1 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:40.578253 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2bca86-e0e8-47b4-9f21-4f1f957508e1 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:40.676021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d30ea-8dce-4117-b48b-e3edd8ede4eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:40.676786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d30ea-8dce-4117-b48b-e3edd8ede4eb service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:40.677069 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d30ea-8dce-4117-b48b-e3edd8ede4eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:40.677814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d30ea-8dce-4117-b48b-e3edd8ede4eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:40.698291 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d30ea-8dce-4117-b48b-e3edd8ede4eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:40.703123 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d30ea-8dce-4117-b48b-e3edd8ede4eb service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:40.739855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 815/1643] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:40.978873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c3ccf-3b62-4185-aa09-bff51998109c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:40.982729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5e5c3ccf-3b62-4185-aa09-bff51998109c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:40.999664 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c3ccf-3b62-4185-aa09-bff51998109c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:41.019891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:41.021006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:41.021586 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:41.036682 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] PATH_INFO: `/v3/users/5cb7b8d14d2747bfabf54a386f92c4e9` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:41.039320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cb7b8d14d2747bfabf54a386f92c4e9)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:41.042006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:41.211928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cb7b8d14d2747bfabf54a386f92c4e9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:41.215085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cb7b8d14d2747bfabf54a386f92c4e9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:41.241930 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cb7b8d14d2747bfabf54a386f92c4e9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:41.262160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cb7b8d14d2747bfabf54a386f92c4e9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:41.298663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cb7b8d14d2747bfabf54a386f92c4e9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:41.325921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 829/1644] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:29:40 2022] DELETE /identity/v3/users/5cb7b8d14d2747bfabf54a386f92c4e9 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:41.330733 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6f89f-116c-4968-9348-f2f33e1f7203 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:41.331017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6f89f-116c-4968-9348-f2f33e1f7203 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:41.331294 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6f89f-116c-4968-9348-f2f33e1f7203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:41.377402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-69a6f89f-116c-4968-9348-f2f33e1f7203 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:41.378651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-69a6f89f-116c-4968-9348-f2f33e1f7203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:41.508559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 816/1645] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:41.539990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:41.542108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:41.641105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ee5f3-d152-4bd3-9261-1267ddb3cd27 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:41.651565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a27a6-d82b-4cf0-ab43-0fbe79add040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:41.653090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a27a6-d82b-4cf0-ab43-0fbe79add040 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:41.653572 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a27a6-d82b-4cf0-ab43-0fbe79add040 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:41.654041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a27a6-d82b-4cf0-ab43-0fbe79add040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:41.673718 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a27a6-d82b-4cf0-ab43-0fbe79add040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:41.675755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a27a6-d82b-4cf0-ab43-0fbe79add040 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:41.713840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 830/1646] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:42.979560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6f89f-116c-4968-9348-f2f33e1f7203 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:42.981684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-69a6f89f-116c-4968-9348-f2f33e1f7203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:42.996448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6f89f-116c-4968-9348-f2f33e1f7203 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:43.002910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:43.004449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:43.004930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:43.005392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] PATH_INFO: `/v3/projects/9c6ae8fc112c4e8a80d458ee4fadb479` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:43.007590 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c6ae8fc112c4e8a80d458ee4fadb479)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:43.012257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:43.081209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:43.098813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c6ae8fc112c4e8a80d458ee4fadb479', 'request_id': 'req-4e86cd37-5f5c-4024-ab58-ff1905c17e23'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.113597 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c6ae8fc112c4e8a80d458ee4fadb479', 'request_id': 'req-4e86cd37-5f5c-4024-ab58-ff1905c17e23'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.127446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c6ae8fc112c4e8a80d458ee4fadb479', 'request_id': 'req-4e86cd37-5f5c-4024-ab58-ff1905c17e23'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.152375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] The token cache is being invalidate because project 9c6ae8fc112c4e8a80d458ee4fadb479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:29:43.152375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.153072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.158601 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 817/1647] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:29:42 2022] DELETE /identity/v3/projects/9c6ae8fc112c4e8a80d458ee4fadb479 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:43.160461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a27a6-d82b-4cf0-ab43-0fbe79add040 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:43.174951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a27a6-d82b-4cf0-ab43-0fbe79add040 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:43.209957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:43.230174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:43.231658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:43.351834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:43.353539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:43.353828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:43.354108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] PATH_INFO: `/v3/users/8eb4ccf85fca43c483b17173496cdf5c` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:43.356257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb4ccf85fca43c483b17173496cdf5c)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:43.363071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:43.406543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86cd37-5f5c-4024-ab58-ff1905c17e23 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:43.424415 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acdbde-8404-42ee-bb00-f407424c2f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:43.425405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acdbde-8404-42ee-bb00-f407424c2f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:43.425717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acdbde-8404-42ee-bb00-f407424c2f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:43.426002 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acdbde-8404-42ee-bb00-f407424c2f38 admin admin] PATH_INFO: `/v3/projects/9394b17f9a8141fea20a24de7bf47d49` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:43.428651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acdbde-8404-42ee-bb00-f407424c2f38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9394b17f9a8141fea20a24de7bf47d49)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:43.430940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acdbde-8404-42ee-bb00-f407424c2f38 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:43.480793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb4ccf85fca43c483b17173496cdf5c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.481516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb4ccf85fca43c483b17173496cdf5c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.497434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb4ccf85fca43c483b17173496cdf5c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.513464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb4ccf85fca43c483b17173496cdf5c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.547591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-80acdbde-8404-42ee-bb00-f407424c2f38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9394b17f9a8141fea20a24de7bf47d49', 'request_id': 'req-80acdbde-8404-42ee-bb00-f407424c2f38'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.560956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb4ccf85fca43c483b17173496cdf5c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.563364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-80acdbde-8404-42ee-bb00-f407424c2f38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9394b17f9a8141fea20a24de7bf47d49', 'request_id': 'req-80acdbde-8404-42ee-bb00-f407424c2f38'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.576150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-80acdbde-8404-42ee-bb00-f407424c2f38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9394b17f9a8141fea20a24de7bf47d49', 'request_id': 'req-80acdbde-8404-42ee-bb00-f407424c2f38'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.583692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 831/1648] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:29:43 2022] DELETE /identity/v3/users/8eb4ccf85fca43c483b17173496cdf5c => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:43.598443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-80acdbde-8404-42ee-bb00-f407424c2f38 admin admin] The token cache is being invalidate because project 9394b17f9a8141fea20a24de7bf47d49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:29:43.599001 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-80acdbde-8404-42ee-bb00-f407424c2f38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.600102 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-80acdbde-8404-42ee-bb00-f407424c2f38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.606672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 818/1649] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:29:43 2022] DELETE /identity/v3/projects/9394b17f9a8141fea20a24de7bf47d49 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:29:43.607159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:43.608702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:43.805970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46839e-b6e0-479a-b21c-24403325edcf admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:43.811335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:43.814896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:43.815237 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:43.815528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] PATH_INFO: `/v3/users/34409e507bd24d5e93bfd4947b74f591` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:43.816886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34409e507bd24d5e93bfd4947b74f591)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:43.818899 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:43.910431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:43.922282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34409e507bd24d5e93bfd4947b74f591'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.923548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34409e507bd24d5e93bfd4947b74f591'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.933888 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34409e507bd24d5e93bfd4947b74f591'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:43.950988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34409e507bd24d5e93bfd4947b74f591'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:44.001298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34409e507bd24d5e93bfd4947b74f591'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:44.023190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 832/1650] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:29:43 2022] DELETE /identity/v3/users/34409e507bd24d5e93bfd4947b74f591 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:44.929877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf9590-0fa8-492e-9c4c-8f5b4828822f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:44.936222 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf9590-0fa8-492e-9c4c-8f5b4828822f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:44.936222 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf9590-0fa8-492e-9c4c-8f5b4828822f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:44.964818 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:44.966667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:45.040840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-30cf9590-0fa8-492e-9c4c-8f5b4828822f None None] MFA Rules not processed for user `54b72a430d9d4d43954eb56d703079b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:45.042099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-30cf9590-0fa8-492e-9c4c-8f5b4828822f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:45.155459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17231a64-f323-4bf9-8b23-8dd2a0407b92 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:45.173207 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fe08d-2740-478a-9c51-eb2d38c44886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:45.174030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fe08d-2740-478a-9c51-eb2d38c44886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:45.175682 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fe08d-2740-478a-9c51-eb2d38c44886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:45.176034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fe08d-2740-478a-9c51-eb2d38c44886 admin admin] PATH_INFO: `/v3/projects/7aff275667bc480a8df0cd1a0a0c1a3c` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:45.182259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fe08d-2740-478a-9c51-eb2d38c44886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aff275667bc480a8df0cd1a0a0c1a3c)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:45.184231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: WARNING py.warnings [None req-490fe08d-2740-478a-9c51-eb2d38c44886 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:29:45.184231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: warnings.warn(msg) Sep 11 05:29:45.184231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: Sep 11 05:29:45.186442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fe08d-2740-478a-9c51-eb2d38c44886 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:45.188664 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:45.249192 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 819/1651] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:45.250937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf9590-0fa8-492e-9c4c-8f5b4828822f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:45.260485 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-30cf9590-0fa8-492e-9c4c-8f5b4828822f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:45.322801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf9590-0fa8-492e-9c4c-8f5b4828822f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:45.354835 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-490fe08d-2740-478a-9c51-eb2d38c44886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aff275667bc480a8df0cd1a0a0c1a3c', 'request_id': 'req-490fe08d-2740-478a-9c51-eb2d38c44886'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:45.362806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-490fe08d-2740-478a-9c51-eb2d38c44886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aff275667bc480a8df0cd1a0a0c1a3c', 'request_id': 'req-490fe08d-2740-478a-9c51-eb2d38c44886'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:45.427252 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-490fe08d-2740-478a-9c51-eb2d38c44886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aff275667bc480a8df0cd1a0a0c1a3c', 'request_id': 'req-490fe08d-2740-478a-9c51-eb2d38c44886'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:45.479070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-490fe08d-2740-478a-9c51-eb2d38c44886 admin admin] The token cache is being invalidate because project 7aff275667bc480a8df0cd1a0a0c1a3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:29:45.479602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-490fe08d-2740-478a-9c51-eb2d38c44886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:45.480595 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-490fe08d-2740-478a-9c51-eb2d38c44886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:45.483438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 833/1652] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:29:44 2022] DELETE /identity/v3/projects/7aff275667bc480a8df0cd1a0a0c1a3c => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:45.485425 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9d12e-2844-4a52-ab46-d9d4443c27c2 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:45.486238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9d12e-2844-4a52-ab46-d9d4443c27c2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:45.487054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9d12e-2844-4a52-ab46-d9d4443c27c2 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:45.491647 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 834/1653] 10.222.0.33 () {54 vars in 962 bytes} [Sun Sep 11 05:29:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:29:45.493985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917db73-1ca3-42ae-9c4c-98c1da11ce83 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:45.494428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917db73-1ca3-42ae-9c4c-98c1da11ce83 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:45.494841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917db73-1ca3-42ae-9c4c-98c1da11ce83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:45.546541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2917db73-1ca3-42ae-9c4c-98c1da11ce83 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:45.549981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2917db73-1ca3-42ae-9c4c-98c1da11ce83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:45.594824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69443f7-57d0-4875-8a9f-789e1a40b447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:45.595497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69443f7-57d0-4875-8a9f-789e1a40b447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:45.595779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69443f7-57d0-4875-8a9f-789e1a40b447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:45.596080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69443f7-57d0-4875-8a9f-789e1a40b447 admin admin] PATH_INFO: `/v3/projects/db63b71df8c547ec8afb2c623f4673d1` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:45.611021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69443f7-57d0-4875-8a9f-789e1a40b447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db63b71df8c547ec8afb2c623f4673d1)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:45.613303 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69443f7-57d0-4875-8a9f-789e1a40b447 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:45.744840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:45.797646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f69443f7-57d0-4875-8a9f-789e1a40b447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db63b71df8c547ec8afb2c623f4673d1', 'request_id': 'req-f69443f7-57d0-4875-8a9f-789e1a40b447'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:45.811855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 835/1654] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:45.822623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd173d6-a875-4497-9cbc-35e5f4383e3b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:45.822811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd173d6-a875-4497-9cbc-35e5f4383e3b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:45.823084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd173d6-a875-4497-9cbc-35e5f4383e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:45.827679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f69443f7-57d0-4875-8a9f-789e1a40b447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db63b71df8c547ec8afb2c623f4673d1', 'request_id': 'req-f69443f7-57d0-4875-8a9f-789e1a40b447'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:45.839431 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f69443f7-57d0-4875-8a9f-789e1a40b447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db63b71df8c547ec8afb2c623f4673d1', 'request_id': 'req-f69443f7-57d0-4875-8a9f-789e1a40b447'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:45.857957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f69443f7-57d0-4875-8a9f-789e1a40b447 admin admin] The token cache is being invalidate because project db63b71df8c547ec8afb2c623f4673d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:29:45.858511 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f69443f7-57d0-4875-8a9f-789e1a40b447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:45.860288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f69443f7-57d0-4875-8a9f-789e1a40b447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:45.866672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 820/1655] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:29:45 2022] DELETE /identity/v3/projects/db63b71df8c547ec8afb2c623f4673d1 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:45.868418 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c718e-771b-42d3-bf58-a70b65ffa491 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:45.868846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c718e-771b-42d3-bf58-a70b65ffa491 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:45.869261 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c718e-771b-42d3-bf58-a70b65ffa491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:45.901136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c18c718e-771b-42d3-bf58-a70b65ffa491 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:45.902758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c18c718e-771b-42d3-bf58-a70b65ffa491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:45.999494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fcd173d6-a875-4497-9cbc-35e5f4383e3b None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:46.001377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fcd173d6-a875-4497-9cbc-35e5f4383e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:46.189744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 821/1656] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:46.189744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc734cea-0bf9-4b40-b226-3ebef61b0c90 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:46.189744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc734cea-0bf9-4b40-b226-3ebef61b0c90 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:46.189744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc734cea-0bf9-4b40-b226-3ebef61b0c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:46.199276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 836/1657] 10.222.0.33 () {58 vars in 1059 bytes} [Sun Sep 11 05:29:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:46.199949 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd173d6-a875-4497-9cbc-35e5f4383e3b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:46.228395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd173d6-a875-4497-9cbc-35e5f4383e3b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:46.243851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-bc734cea-0bf9-4b40-b226-3ebef61b0c90 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:46.244739 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25029a31-a75d-49d8-8b42-4ab5d09acfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:46.245279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bc734cea-0bf9-4b40-b226-3ebef61b0c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:46.245382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25029a31-a75d-49d8-8b42-4ab5d09acfdf admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:46.245660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25029a31-a75d-49d8-8b42-4ab5d09acfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:46.245937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25029a31-a75d-49d8-8b42-4ab5d09acfdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:46.246608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25029a31-a75d-49d8-8b42-4ab5d09acfdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:29:46.247242 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25029a31-a75d-49d8-8b42-4ab5d09acfdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:46.248646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25029a31-a75d-49d8-8b42-4ab5d09acfdf admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:46.283804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 837/1658] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:29:46.284729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25029a31-a75d-49d8-8b42-4ab5d09acfdf admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:46.285969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 822/1659] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:46.292633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712749c-18da-48fc-964b-6c1f7263aef8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:46.293189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712749c-18da-48fc-964b-6c1f7263aef8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:46.293644 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712749c-18da-48fc-964b-6c1f7263aef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:46.321106 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25029a31-a75d-49d8-8b42-4ab5d09acfdf admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:46.341181 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a712749c-18da-48fc-964b-6c1f7263aef8 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:46.344877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a712749c-18da-48fc-964b-6c1f7263aef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:46.345724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468279b-ae9c-4a98-8dd2-825d7f453b53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:46.346430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468279b-ae9c-4a98-8dd2-825d7f453b53 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:46.346718 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468279b-ae9c-4a98-8dd2-825d7f453b53 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:46.346995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468279b-ae9c-4a98-8dd2-825d7f453b53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:46.358826 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7468279b-ae9c-4a98-8dd2-825d7f453b53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:46.370514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 823/1660] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:46.372965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712749c-18da-48fc-964b-6c1f7263aef8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:46.384570 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:46.403065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712749c-18da-48fc-964b-6c1f7263aef8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:46.415614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc367802-decd-4973-b573-cffd13220ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:46.416267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc367802-decd-4973-b573-cffd13220ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:46.416552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc367802-decd-4973-b573-cffd13220ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:46.416826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc367802-decd-4973-b573-cffd13220ea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:46.417480 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc367802-decd-4973-b573-cffd13220ea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:46.429110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc367802-decd-4973-b573-cffd13220ea0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:46.475854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468279b-ae9c-4a98-8dd2-825d7f453b53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:46.477671 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468279b-ae9c-4a98-8dd2-825d7f453b53 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:46.489506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 838/1661] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:46.490590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468279b-ae9c-4a98-8dd2-825d7f453b53 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:46.505195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468279b-ae9c-4a98-8dd2-825d7f453b53 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:46.511548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f868b7-c3c0-48f9-a8de-9f4e542de4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:46.512775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f868b7-c3c0-48f9-a8de-9f4e542de4e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:46.513201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f868b7-c3c0-48f9-a8de-9f4e542de4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:46.513654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f868b7-c3c0-48f9-a8de-9f4e542de4e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:46.514514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f868b7-c3c0-48f9-a8de-9f4e542de4e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:29:46.514962 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f868b7-c3c0-48f9-a8de-9f4e542de4e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:46.519137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f868b7-c3c0-48f9-a8de-9f4e542de4e2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:46.526812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 824/1662] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:46 2022] POST /identity/v3/projects => generated 365 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:46.527799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc367802-decd-4973-b573-cffd13220ea0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:46.536748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 839/1663] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:29:46.540983 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f868b7-c3c0-48f9-a8de-9f4e542de4e2 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:46.556893 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f868b7-c3c0-48f9-a8de-9f4e542de4e2 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:46.653621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:46.695153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d903d4-0d57-4118-b7a0-6d18716330da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:46.696166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d903d4-0d57-4118-b7a0-6d18716330da admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:46.696274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d903d4-0d57-4118-b7a0-6d18716330da admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:46.696518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d903d4-0d57-4118-b7a0-6d18716330da admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:46.697279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d903d4-0d57-4118-b7a0-6d18716330da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:46.699242 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d903d4-0d57-4118-b7a0-6d18716330da admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:46.735499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc367802-decd-4973-b573-cffd13220ea0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:46.754573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339f6bc-492f-4299-8c74-2303f01856c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:46.755144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339f6bc-492f-4299-8c74-2303f01856c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:46.755629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339f6bc-492f-4299-8c74-2303f01856c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:46.755991 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339f6bc-492f-4299-8c74-2303f01856c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:46.758362 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339f6bc-492f-4299-8c74-2303f01856c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:46.761695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 840/1664] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:46 2022] POST /identity/v3/projects => generated 363 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:46.764487 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339f6bc-492f-4299-8c74-2303f01856c3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:46.766141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d903d4-0d57-4118-b7a0-6d18716330da admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:46.829436 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 825/1665] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:46 2022] POST /identity/v3/users => generated 283 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:46.833561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339f6bc-492f-4299-8c74-2303f01856c3 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:46.862574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339f6bc-492f-4299-8c74-2303f01856c3 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:46.903048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d903d4-0d57-4118-b7a0-6d18716330da admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:46.914244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4884480-2614-404a-8188-3be148aba195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:46.914884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4884480-2614-404a-8188-3be148aba195 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:46.915161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4884480-2614-404a-8188-3be148aba195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:46.915437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4884480-2614-404a-8188-3be148aba195 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:46.919607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4884480-2614-404a-8188-3be148aba195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:46.927337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4884480-2614-404a-8188-3be148aba195 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:46.974779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 841/1666] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:46 2022] POST /identity/v3/users => generated 282 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:46.978197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4884480-2614-404a-8188-3be148aba195 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:46.994204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4653b0-5895-4fbe-b789-b471427f25c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:46.994851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4653b0-5895-4fbe-b789-b471427f25c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:46.997329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4653b0-5895-4fbe-b789-b471427f25c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:46.997635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4653b0-5895-4fbe-b789-b471427f25c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:47.001412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4653b0-5895-4fbe-b789-b471427f25c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:47.003118 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4653b0-5895-4fbe-b789-b471427f25c2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:47.006120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4884480-2614-404a-8188-3be148aba195 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:47.026818 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472a237-715e-4b82-9d2d-4a4a660009a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:47.029199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472a237-715e-4b82-9d2d-4a4a660009a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:47.029499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472a237-715e-4b82-9d2d-4a4a660009a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:47.029779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472a237-715e-4b82-9d2d-4a4a660009a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:47.031144 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472a237-715e-4b82-9d2d-4a4a660009a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:47.036456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472a237-715e-4b82-9d2d-4a4a660009a3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:47.038512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 826/1667] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:47.042263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4653b0-5895-4fbe-b789-b471427f25c2 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:47.063306 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 842/1668] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:47.066685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472a237-715e-4b82-9d2d-4a4a660009a3 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:47.069638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4653b0-5895-4fbe-b789-b471427f25c2 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:47.086606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a13eb-7254-4463-bc66-8c429e2eafa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:47.088787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a13eb-7254-4463-bc66-8c429e2eafa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:47.091579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a13eb-7254-4463-bc66-8c429e2eafa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:47.091818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a13eb-7254-4463-bc66-8c429e2eafa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:47.092555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a13eb-7254-4463-bc66-8c429e2eafa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:47.094338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a13eb-7254-4463-bc66-8c429e2eafa1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:47.101180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472a237-715e-4b82-9d2d-4a4a660009a3 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:47.113105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a20a2-c1f8-4cdf-acb0-7aad41e91c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:47.115711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a20a2-c1f8-4cdf-acb0-7aad41e91c7f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:47.116143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a20a2-c1f8-4cdf-acb0-7aad41e91c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:47.116431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a20a2-c1f8-4cdf-acb0-7aad41e91c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:47.117401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a20a2-c1f8-4cdf-acb0-7aad41e91c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:47.121944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a20a2-c1f8-4cdf-acb0-7aad41e91c7f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:47.132219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 827/1669] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:47.136727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a13eb-7254-4463-bc66-8c429e2eafa1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:47.168504 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 843/1670] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:47.175221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a20a2-c1f8-4cdf-acb0-7aad41e91c7f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:47.176055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a13eb-7254-4463-bc66-8c429e2eafa1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:47.186308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c4ddc-a6c0-457f-8dab-f212dd0828db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:47.187002 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c4ddc-a6c0-457f-8dab-f212dd0828db admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:47.193821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c4ddc-a6c0-457f-8dab-f212dd0828db admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:47.194474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c4ddc-a6c0-457f-8dab-f212dd0828db admin admin] PATH_INFO: `/v3/projects/2aec32d2b6714424a213b0d6cde5354a/users/5b02a00dce074792bafeb433d0ab4aa1/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:47.205072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a20a2-c1f8-4cdf-acb0-7aad41e91c7f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:47.220850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6eb05a-ad63-4db9-8516-093f3e23c31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:47.221550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6eb05a-ad63-4db9-8516-093f3e23c31c admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:47.221859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6eb05a-ad63-4db9-8516-093f3e23c31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:47.222143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6eb05a-ad63-4db9-8516-093f3e23c31c admin admin] PATH_INFO: `/v3/projects/31fb04878484436380a4064e97ed9763/users/cc04ad9577ae42b786b3a3c8da3ebb95/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:47.240667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c4ddc-a6c0-457f-8dab-f212dd0828db admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aec32d2b6714424a213b0d6cde5354a, user_id=5b02a00dce074792bafeb433d0ab4aa1, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:47.246172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c4ddc-a6c0-457f-8dab-f212dd0828db admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:47.269686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6eb05a-ad63-4db9-8516-093f3e23c31c admin admin] RBAC: Authorizing `identity:create_grant(project_id=31fb04878484436380a4064e97ed9763, user_id=cc04ad9577ae42b786b3a3c8da3ebb95, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:47.279657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6eb05a-ad63-4db9-8516-093f3e23c31c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:47.280444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 828/1671] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:47 2022] PUT /identity/v3/projects/2aec32d2b6714424a213b0d6cde5354a/users/5b02a00dce074792bafeb433d0ab4aa1/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:47.294013 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e3395-2e16-4192-b284-eb9b86919410 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:47.294312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e3395-2e16-4192-b284-eb9b86919410 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:47.294706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e3395-2e16-4192-b284-eb9b86919410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:47.308558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 844/1672] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:47 2022] PUT /identity/v3/projects/31fb04878484436380a4064e97ed9763/users/cc04ad9577ae42b786b3a3c8da3ebb95/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:47.318413 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467032c-c196-4bd2-8372-b3547a12037d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:47.318849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467032c-c196-4bd2-8372-b3547a12037d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:47.319260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467032c-c196-4bd2-8372-b3547a12037d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:47.360341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b66e3395-2e16-4192-b284-eb9b86919410 None None] MFA Rules not processed for user `5b02a00dce074792bafeb433d0ab4aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:47.371283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b66e3395-2e16-4192-b284-eb9b86919410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:47.415732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b467032c-c196-4bd2-8372-b3547a12037d None None] MFA Rules not processed for user `cc04ad9577ae42b786b3a3c8da3ebb95`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:47.431366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b467032c-c196-4bd2-8372-b3547a12037d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:47.450086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:47.563293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 829/1673] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:47.566389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e3395-2e16-4192-b284-eb9b86919410 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:47.596608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:47.622145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 845/1674] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:29:47.623893 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467032c-c196-4bd2-8372-b3547a12037d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:47.811623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e3395-2e16-4192-b284-eb9b86919410 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:47.827008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317a5c1-c0fd-47a9-a473-4a67cb45e536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:47.827625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467032c-c196-4bd2-8372-b3547a12037d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:47.828336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317a5c1-c0fd-47a9-a473-4a67cb45e536 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:47.831319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317a5c1-c0fd-47a9-a473-4a67cb45e536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:47.833233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317a5c1-c0fd-47a9-a473-4a67cb45e536 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:47.836369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317a5c1-c0fd-47a9-a473-4a67cb45e536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:47.837259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3530a-e8e7-4351-8bb5-dd88b044baf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:47.837998 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3530a-e8e7-4351-8bb5-dd88b044baf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:47.839577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3530a-e8e7-4351-8bb5-dd88b044baf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:47.839967 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3530a-e8e7-4351-8bb5-dd88b044baf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:47.842142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317a5c1-c0fd-47a9-a473-4a67cb45e536 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:47.842377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3530a-e8e7-4351-8bb5-dd88b044baf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:47.848076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3530a-e8e7-4351-8bb5-dd88b044baf7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:47.911542 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 846/1675] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:47 2022] POST /identity/v3/projects => generated 365 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:47.914863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 830/1676] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:29:47 2022] POST /identity/v3/projects => generated 363 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:47.915523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3530a-e8e7-4351-8bb5-dd88b044baf7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:47.917296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317a5c1-c0fd-47a9-a473-4a67cb45e536 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:48.033691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3530a-e8e7-4351-8bb5-dd88b044baf7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:48.072259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317a5c1-c0fd-47a9-a473-4a67cb45e536 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:48.083641 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9625f-76a7-431d-ba32-8d46551da30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:48.084221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9625f-76a7-431d-ba32-8d46551da30a admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:48.084504 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9625f-76a7-431d-ba32-8d46551da30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:48.084785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9625f-76a7-431d-ba32-8d46551da30a admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:48.085427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9625f-76a7-431d-ba32-8d46551da30a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:48.092110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9625f-76a7-431d-ba32-8d46551da30a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:48.144894 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 831/1677] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:47 2022] POST /identity/v3/users => generated 288 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:48.146377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9625f-76a7-431d-ba32-8d46551da30a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:48.168858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9625f-76a7-431d-ba32-8d46551da30a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:48.179915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7a6b8-722a-45a5-9063-5b440119ee79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:48.180146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc4256-7c5f-453e-9121-bf57f477bb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:48.180734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc4256-7c5f-453e-9121-bf57f477bb11 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:48.180883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7a6b8-722a-45a5-9063-5b440119ee79 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:48.181016 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc4256-7c5f-453e-9121-bf57f477bb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:48.181289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc4256-7c5f-453e-9121-bf57f477bb11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:48.181395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7a6b8-722a-45a5-9063-5b440119ee79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:48.181700 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7a6b8-722a-45a5-9063-5b440119ee79 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:48.181945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc4256-7c5f-453e-9121-bf57f477bb11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:48.182462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7a6b8-722a-45a5-9063-5b440119ee79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:48.183573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc4256-7c5f-453e-9121-bf57f477bb11 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:48.186168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7a6b8-722a-45a5-9063-5b440119ee79 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:48.228030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 832/1678] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:48.229129 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc4256-7c5f-453e-9121-bf57f477bb11 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:48.236525 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 847/1679] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:29:47 2022] POST /identity/v3/users => generated 289 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:29:48.239055 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7a6b8-722a-45a5-9063-5b440119ee79 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:48.253470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc4256-7c5f-453e-9121-bf57f477bb11 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:48.255252 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7a6b8-722a-45a5-9063-5b440119ee79 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:48.259160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68caa7-2d02-429d-a67b-cef5d71e9a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:48.259856 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68caa7-2d02-429d-a67b-cef5d71e9a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:48.260176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68caa7-2d02-429d-a67b-cef5d71e9a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:48.260456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68caa7-2d02-429d-a67b-cef5d71e9a7c admin admin] PATH_INFO: `/v3/projects/98fdca1edcf4442b86dbc19bc3612547/users/dc1a61ce00d04103bfec53a16cd41cfd/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:48.261262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead25d5-2264-437f-a95e-8fd189ffaa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:48.262456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead25d5-2264-437f-a95e-8fd189ffaa8a admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:48.262739 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead25d5-2264-437f-a95e-8fd189ffaa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:48.263010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead25d5-2264-437f-a95e-8fd189ffaa8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:48.264362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead25d5-2264-437f-a95e-8fd189ffaa8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:48.267428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead25d5-2264-437f-a95e-8fd189ffaa8a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:48.304906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 848/1680] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:48.305356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68caa7-2d02-429d-a67b-cef5d71e9a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=98fdca1edcf4442b86dbc19bc3612547, user_id=dc1a61ce00d04103bfec53a16cd41cfd, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:48.310112 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead25d5-2264-437f-a95e-8fd189ffaa8a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:48.311639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68caa7-2d02-429d-a67b-cef5d71e9a7c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:48.335460 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:48.335632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead25d5-2264-437f-a95e-8fd189ffaa8a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:48.344291 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 833/1681] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:48 2022] PUT /identity/v3/projects/98fdca1edcf4442b86dbc19bc3612547/users/dc1a61ce00d04103bfec53a16cd41cfd/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:48.348350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68caa7-2d02-429d-a67b-cef5d71e9a7c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:48.353921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:48.461762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68caa7-2d02-429d-a67b-cef5d71e9a7c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:48.467259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf4c5f-0395-4ed9-be52-69ba9e919406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:48.468178 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf4c5f-0395-4ed9-be52-69ba9e919406 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:48.468591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf4c5f-0395-4ed9-be52-69ba9e919406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:48.468995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf4c5f-0395-4ed9-be52-69ba9e919406 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:48.470237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf4c5f-0395-4ed9-be52-69ba9e919406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:48.472060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf4c5f-0395-4ed9-be52-69ba9e919406 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:48.505607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 834/1682] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:48.509822 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af45cbf-2e1d-4c36-9e68-c3be2d04253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:48.510800 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af45cbf-2e1d-4c36-9e68-c3be2d04253b admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:48.511226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af45cbf-2e1d-4c36-9e68-c3be2d04253b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:48.511583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af45cbf-2e1d-4c36-9e68-c3be2d04253b admin admin] PATH_INFO: `/v3/projects/46db4ed6e4a04c81ab9a7b338c523d43/users/a841ac78e41e4611bd0f18bb818c9d18/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:48.511759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf4c5f-0395-4ed9-be52-69ba9e919406 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:48.526632 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf4c5f-0395-4ed9-be52-69ba9e919406 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:48.534309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25071782-873d-4521-902e-b9a4d1297625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:48.535356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25071782-873d-4521-902e-b9a4d1297625 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:48.535897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25071782-873d-4521-902e-b9a4d1297625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:48.536268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25071782-873d-4521-902e-b9a4d1297625 admin admin] PATH_INFO: `/v3/projects/98fdca1edcf4442b86dbc19bc3612547/users/dc1a61ce00d04103bfec53a16cd41cfd/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:48.539437 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25071782-873d-4521-902e-b9a4d1297625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98fdca1edcf4442b86dbc19bc3612547, user_id=dc1a61ce00d04103bfec53a16cd41cfd, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:48.544041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af45cbf-2e1d-4c36-9e68-c3be2d04253b admin admin] RBAC: Authorizing `identity:create_grant(project_id=46db4ed6e4a04c81ab9a7b338c523d43, user_id=a841ac78e41e4611bd0f18bb818c9d18, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:48.544869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25071782-873d-4521-902e-b9a4d1297625 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:48.552959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af45cbf-2e1d-4c36-9e68-c3be2d04253b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:48.582027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 835/1683] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:48 2022] PUT /identity/v3/projects/98fdca1edcf4442b86dbc19bc3612547/users/dc1a61ce00d04103bfec53a16cd41cfd/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:48.587839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dde0d-ce44-4e4c-a06e-17f6908cd4db None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:48.588188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dde0d-ce44-4e4c-a06e-17f6908cd4db None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:48.588560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776dde0d-ce44-4e4c-a06e-17f6908cd4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:48.632993 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 849/1684] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:48 2022] PUT /identity/v3/projects/46db4ed6e4a04c81ab9a7b338c523d43/users/a841ac78e41e4611bd0f18bb818c9d18/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:48.635236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af45cbf-2e1d-4c36-9e68-c3be2d04253b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:48.660764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-776dde0d-ce44-4e4c-a06e-17f6908cd4db None None] MFA Rules not processed for user `dc1a61ce00d04103bfec53a16cd41cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:48.662283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-776dde0d-ce44-4e4c-a06e-17f6908cd4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:48.818346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af45cbf-2e1d-4c36-9e68-c3be2d04253b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:48.834301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafef08-8660-4792-a03c-b0ffa982525e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:48.834940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafef08-8660-4792-a03c-b0ffa982525e admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:48.835217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafef08-8660-4792-a03c-b0ffa982525e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:48.835485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafef08-8660-4792-a03c-b0ffa982525e admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:48.836185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafef08-8660-4792-a03c-b0ffa982525e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:48.837895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafef08-8660-4792-a03c-b0ffa982525e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:48.865700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 836/1685] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:48.881090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776dde0d-ce44-4e4c-a06e-17f6908cd4db None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:48.893637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-776dde0d-ce44-4e4c-a06e-17f6908cd4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:48.899890 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 850/1686] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:29:48.901173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafef08-8660-4792-a03c-b0ffa982525e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:48.935596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafef08-8660-4792-a03c-b0ffa982525e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:48.950044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf3b26-c87e-4988-8874-3c5867c73767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:48.951231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf3b26-c87e-4988-8874-3c5867c73767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:48.951231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf3b26-c87e-4988-8874-3c5867c73767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:48.951678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf3b26-c87e-4988-8874-3c5867c73767 admin admin] PATH_INFO: `/v3/projects/46db4ed6e4a04c81ab9a7b338c523d43/users/a841ac78e41e4611bd0f18bb818c9d18/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:48.955842 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf3b26-c87e-4988-8874-3c5867c73767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46db4ed6e4a04c81ab9a7b338c523d43, user_id=a841ac78e41e4611bd0f18bb818c9d18, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:48.962988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf3b26-c87e-4988-8874-3c5867c73767 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:48.988361 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 851/1687] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:29:48 2022] PUT /identity/v3/projects/46db4ed6e4a04c81ab9a7b338c523d43/users/a841ac78e41e4611bd0f18bb818c9d18/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:49.002708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7633455-351d-4e0d-9709-a75918036687 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:49.003090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7633455-351d-4e0d-9709-a75918036687 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:49.003678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7633455-351d-4e0d-9709-a75918036687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:49.062998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776dde0d-ce44-4e4c-a06e-17f6908cd4db None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:49.089846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d7633455-351d-4e0d-9709-a75918036687 None None] MFA Rules not processed for user `a841ac78e41e4611bd0f18bb818c9d18`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:49.091685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d7633455-351d-4e0d-9709-a75918036687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:49.127369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:49.210366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316a103-2e6a-47fa-97bf-caed50ca258e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:49.210998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316a103-2e6a-47fa-97bf-caed50ca258e service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:49.211278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316a103-2e6a-47fa-97bf-caed50ca258e service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:49.211551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316a103-2e6a-47fa-97bf-caed50ca258e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:49.260976 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:49.273120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 852/1688] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:49.285540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7633455-351d-4e0d-9709-a75918036687 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:49.288172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d7633455-351d-4e0d-9709-a75918036687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:49.313380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7633455-351d-4e0d-9709-a75918036687 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:49.329837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbbf54-7ceb-47b5-b6f1-79b04e7f1c0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:49.330768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbbf54-7ceb-47b5-b6f1-79b04e7f1c0f service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:49.331251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbbf54-7ceb-47b5-b6f1-79b04e7f1c0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:49.332078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbbf54-7ceb-47b5-b6f1-79b04e7f1c0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:49.358033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccbbf54-7ceb-47b5-b6f1-79b04e7f1c0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:49.360527 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccbbf54-7ceb-47b5-b6f1-79b04e7f1c0f service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:49.386825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316a103-2e6a-47fa-97bf-caed50ca258e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:49.388266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 853/1689] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:29:49.388888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316a103-2e6a-47fa-97bf-caed50ca258e service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:49.404998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 837/1690] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:49.569791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbbf54-7ceb-47b5-b6f1-79b04e7f1c0f service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:49.579801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbbf54-7ceb-47b5-b6f1-79b04e7f1c0f service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:49.585440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316a103-2e6a-47fa-97bf-caed50ca258e service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:49.587990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1627f6f-4e86-4ed5-bd93-8c8bd1b25445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:49.588904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1627f6f-4e86-4ed5-bd93-8c8bd1b25445 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:49.589389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1627f6f-4e86-4ed5-bd93-8c8bd1b25445 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:49.589886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1627f6f-4e86-4ed5-bd93-8c8bd1b25445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:49.600041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316a103-2e6a-47fa-97bf-caed50ca258e service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:49.610936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2634209-eddd-42f8-8182-7a17b7ed9171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:49.611548 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2634209-eddd-42f8-8182-7a17b7ed9171 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:49.611825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2634209-eddd-42f8-8182-7a17b7ed9171 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:49.613895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2634209-eddd-42f8-8182-7a17b7ed9171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:49.705369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1627f6f-4e86-4ed5-bd93-8c8bd1b25445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:49.705369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1627f6f-4e86-4ed5-bd93-8c8bd1b25445 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:49.714125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 854/1691] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:49.715859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef486a73-7114-4c66-9ca3-e26b73e62544 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:49.716302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef486a73-7114-4c66-9ca3-e26b73e62544 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:49.716687 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef486a73-7114-4c66-9ca3-e26b73e62544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:49.757268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ef486a73-7114-4c66-9ca3-e26b73e62544 None None] MFA Rules not processed for user `f4c978e96d3248d59120c287ecc75e87`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:49.758921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ef486a73-7114-4c66-9ca3-e26b73e62544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:49.833407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2634209-eddd-42f8-8182-7a17b7ed9171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:49.834890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2634209-eddd-42f8-8182-7a17b7ed9171 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:49.867746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 838/1692] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:49.937161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 855/1693] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:49.954428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a83e31-28e2-4519-bdc5-c4e874ce9635 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:49.955199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a83e31-28e2-4519-bdc5-c4e874ce9635 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:49.956169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a83e31-28e2-4519-bdc5-c4e874ce9635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:50.021334 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-34a83e31-28e2-4519-bdc5-c4e874ce9635 None None] MFA Rules not processed for user `f4c978e96d3248d59120c287ecc75e87`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:50.023449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-34a83e31-28e2-4519-bdc5-c4e874ce9635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:50.040860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 839/1694] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:50.088246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3120a4-d065-40e3-a677-9489a46c7619 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:50.088590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3120a4-d065-40e3-a677-9489a46c7619 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:50.088922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3120a4-d065-40e3-a677-9489a46c7619 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:50.090724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 856/1695] 10.222.0.33 () {54 vars in 962 bytes} [Sun Sep 11 05:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:29:50.154494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec095f6f-dd6a-4d1d-8f97-8eb6ef7bb104 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:50.155200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec095f6f-dd6a-4d1d-8f97-8eb6ef7bb104 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:50.155678 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec095f6f-dd6a-4d1d-8f97-8eb6ef7bb104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:50.203618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ec095f6f-dd6a-4d1d-8f97-8eb6ef7bb104 None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:50.207603 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ec095f6f-dd6a-4d1d-8f97-8eb6ef7bb104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:50.402337 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:50.420942 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 840/1696] 10.222.0.33 () {58 vars in 1059 bytes} [Sun Sep 11 05:29:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:50.430648 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3120a4-d065-40e3-a677-9489a46c7619 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:50.457406 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3120a4-d065-40e3-a677-9489a46c7619 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:50.464974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9ff99-8ee9-4c29-9ae8-592ae5d58e5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:50.465931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9ff99-8ee9-4c29-9ae8-592ae5d58e5e service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:50.466585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9ff99-8ee9-4c29-9ae8-592ae5d58e5e service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:50.467102 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9ff99-8ee9-4c29-9ae8-592ae5d58e5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:50.483141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9ff99-8ee9-4c29-9ae8-592ae5d58e5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:50.488409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9ff99-8ee9-4c29-9ae8-592ae5d58e5e service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:50.505345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 857/1697] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:50.925460 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8fcf5-42eb-43e5-babc-329ea453cd2a None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:50.926063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8fcf5-42eb-43e5-babc-329ea453cd2a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:50.926464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8fcf5-42eb-43e5-babc-329ea453cd2a None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:50.928371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 841/1698] 10.222.0.33 () {54 vars in 912 bytes} [Sun Sep 11 05:29:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:29:50.934188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4f5ce-9b26-4f87-aabd-4f7fb7e22a93 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:50.934473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4f5ce-9b26-4f87-aabd-4f7fb7e22a93 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:50.934747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4f5ce-9b26-4f87-aabd-4f7fb7e22a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:50.965377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e3f4f5ce-9b26-4f87-aabd-4f7fb7e22a93 None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:50.967073 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e3f4f5ce-9b26-4f87-aabd-4f7fb7e22a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:50.978868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 858/1699] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 05:29:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:50.994985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8fcf5-42eb-43e5-babc-329ea453cd2a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:50.997142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d4e8fcf5-42eb-43e5-babc-329ea453cd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:51.247533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8fcf5-42eb-43e5-babc-329ea453cd2a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:51.259787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9a9fc-ec01-436d-8c1e-a921e95a4bcc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0330c79c391d463b91f99e030c6dcc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:51.260993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9a9fc-ec01-436d-8c1e-a921e95a4bcc service glance] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:51.261280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9a9fc-ec01-436d-8c1e-a921e95a4bcc service glance] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:51.261557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9a9fc-ec01-436d-8c1e-a921e95a4bcc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:51.298176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4f5ce-9b26-4f87-aabd-4f7fb7e22a93 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:51.305766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9a9fc-ec01-436d-8c1e-a921e95a4bcc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:51.307368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9a9fc-ec01-436d-8c1e-a921e95a4bcc service glance] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:51.310560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4f5ce-9b26-4f87-aabd-4f7fb7e22a93 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:51.323565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374297e-ee14-4903-9803-4201628c7ec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:51.324210 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374297e-ee14-4903-9803-4201628c7ec2 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:51.324522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374297e-ee14-4903-9803-4201628c7ec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:51.324924 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374297e-ee14-4903-9803-4201628c7ec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:51.330102 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e374297e-ee14-4903-9803-4201628c7ec2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:51.341310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 842/1700] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:51.524602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:51.568199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374297e-ee14-4903-9803-4201628c7ec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:51.570830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374297e-ee14-4903-9803-4201628c7ec2 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:51.596762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 859/1701] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:29:52.384734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37090cb8-f450-43d6-957c-ebfac55be3d9 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:52.385693 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37090cb8-f450-43d6-957c-ebfac55be3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:52.386120 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37090cb8-f450-43d6-957c-ebfac55be3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:52.431795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-37090cb8-f450-43d6-957c-ebfac55be3d9 None None] MFA Rules not processed for user `54b72a430d9d4d43954eb56d703079b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:52.435967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-37090cb8-f450-43d6-957c-ebfac55be3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:52.600281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 843/1702] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:52.706853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9ad2d-760c-4200-9940-805c1d6a2357 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:52.708805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9ad2d-760c-4200-9940-805c1d6a2357 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:52.710433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9ad2d-760c-4200-9940-805c1d6a2357 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:52.712978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 860/1703] 10.222.0.33 () {54 vars in 912 bytes} [Sun Sep 11 05:29:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:29:52.718818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1249e23-03f6-4dd5-bb0d-206febb1e9c8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:52.719362 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1249e23-03f6-4dd5-bb0d-206febb1e9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:52.719854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1249e23-03f6-4dd5-bb0d-206febb1e9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:52.766835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f1249e23-03f6-4dd5-bb0d-206febb1e9c8 None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:52.770641 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f1249e23-03f6-4dd5-bb0d-206febb1e9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:52.792926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 844/1704] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 05:29:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:29:52.800824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9ad2d-760c-4200-9940-805c1d6a2357 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:52.803793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7ae9ad2d-760c-4200-9940-805c1d6a2357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:52.823814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9ad2d-760c-4200-9940-805c1d6a2357 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:52.829838 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67025d-1b58-4d0e-86e8-bf36bdd6e3ed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0330c79c391d463b91f99e030c6dcc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:52.830686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67025d-1b58-4d0e-86e8-bf36bdd6e3ed service glance] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:52.831103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67025d-1b58-4d0e-86e8-bf36bdd6e3ed service glance] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:52.831511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67025d-1b58-4d0e-86e8-bf36bdd6e3ed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:52.869449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67025d-1b58-4d0e-86e8-bf36bdd6e3ed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:52.871158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67025d-1b58-4d0e-86e8-bf36bdd6e3ed service glance] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:52.891575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 861/1705] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:53.387792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab1ad5-481e-4bcb-b49e-53c27a0a5734 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:53.388323 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab1ad5-481e-4bcb-b49e-53c27a0a5734 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:53.388607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab1ad5-481e-4bcb-b49e-53c27a0a5734 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:53.390650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 845/1706] 10.222.0.33 () {54 vars in 912 bytes} [Sun Sep 11 05:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:29:53.398377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203a627-f082-4e2e-b007-067acfdbf95f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:53.402898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203a627-f082-4e2e-b007-067acfdbf95f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:53.403431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203a627-f082-4e2e-b007-067acfdbf95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:53.483545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f203a627-f082-4e2e-b007-067acfdbf95f None None] MFA Rules not processed for user `0bf2a5de0e744505a1317ce9fa224643`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:53.485154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f203a627-f082-4e2e-b007-067acfdbf95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:53.609738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 862/1707] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 05:29:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:53.637640 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a5aab-1d94-4f66-b381-3fc808a51ebc None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:53.638093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a5aab-1d94-4f66-b381-3fc808a51ebc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:53.638484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a5aab-1d94-4f66-b381-3fc808a51ebc None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:53.639928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 846/1708] 10.222.0.33 () {54 vars in 912 bytes} [Sun Sep 11 05:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:29:53.647313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fa86d-e7dd-45de-83bf-e651c33c25e7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:53.647925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fa86d-e7dd-45de-83bf-e651c33c25e7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:53.648471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fa86d-e7dd-45de-83bf-e651c33c25e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:53.684696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ba7fa86d-e7dd-45de-83bf-e651c33c25e7 None None] MFA Rules not processed for user `0bf2a5de0e744505a1317ce9fa224643`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:53.686951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ba7fa86d-e7dd-45de-83bf-e651c33c25e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:53.706245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 863/1709] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 05:29:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:53.728264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a5aab-1d94-4f66-b381-3fc808a51ebc None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:53.731485 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a13a5aab-1d94-4f66-b381-3fc808a51ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:53.744939 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fa86d-e7dd-45de-83bf-e651c33c25e7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:53.746897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ba7fa86d-e7dd-45de-83bf-e651c33c25e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:53.767512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fa86d-e7dd-45de-83bf-e651c33c25e7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:53.771410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a5aab-1d94-4f66-b381-3fc808a51ebc None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:53.782204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d78d5-643a-46ff-bb9d-275487b89537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:53.782806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d78d5-643a-46ff-bb9d-275487b89537 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:53.783084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d78d5-643a-46ff-bb9d-275487b89537 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:53.783358 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d78d5-643a-46ff-bb9d-275487b89537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:53.784514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da33800-48f7-4bc3-b849-0bfedcd2d293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:53.785126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da33800-48f7-4bc3-b849-0bfedcd2d293 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:53.785400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da33800-48f7-4bc3-b849-0bfedcd2d293 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:53.785670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da33800-48f7-4bc3-b849-0bfedcd2d293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:53.811602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da33800-48f7-4bc3-b849-0bfedcd2d293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:53.813455 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da33800-48f7-4bc3-b849-0bfedcd2d293 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:53.816985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d78d5-643a-46ff-bb9d-275487b89537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:53.818584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d78d5-643a-46ff-bb9d-275487b89537 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:53.838252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 847/1710] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:53.841923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 864/1711] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:29:54.462856 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d78d5-643a-46ff-bb9d-275487b89537 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:54.661656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d78d5-643a-46ff-bb9d-275487b89537 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:54.682427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa437c-912c-4e74-ba9f-02548ce2edf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:54.683206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa437c-912c-4e74-ba9f-02548ce2edf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:54.683542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa437c-912c-4e74-ba9f-02548ce2edf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:54.684617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa437c-912c-4e74-ba9f-02548ce2edf3 admin admin] PATH_INFO: `/v3/users/cc04ad9577ae42b786b3a3c8da3ebb95` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:54.690561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa437c-912c-4e74-ba9f-02548ce2edf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc04ad9577ae42b786b3a3c8da3ebb95)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:54.692363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: WARNING py.warnings [None req-f6fa437c-912c-4e74-ba9f-02548ce2edf3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 05:29:54.692363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: warnings.warn(msg) Sep 11 05:29:54.692363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: Sep 11 05:29:54.693954 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa437c-912c-4e74-ba9f-02548ce2edf3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:54.704865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:54.867260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da33800-48f7-4bc3-b849-0bfedcd2d293 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:54.869555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8da33800-48f7-4bc3-b849-0bfedcd2d293 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:54.947294 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f6fa437c-912c-4e74-ba9f-02548ce2edf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc04ad9577ae42b786b3a3c8da3ebb95'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:54.992259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f6fa437c-912c-4e74-ba9f-02548ce2edf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc04ad9577ae42b786b3a3c8da3ebb95'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:55.076347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f6fa437c-912c-4e74-ba9f-02548ce2edf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc04ad9577ae42b786b3a3c8da3ebb95'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:55.080154 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f6fa437c-912c-4e74-ba9f-02548ce2edf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc04ad9577ae42b786b3a3c8da3ebb95'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:55.096711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da33800-48f7-4bc3-b849-0bfedcd2d293 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:55.105573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-f6fa437c-912c-4e74-ba9f-02548ce2edf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc04ad9577ae42b786b3a3c8da3ebb95'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:55.116548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db2986-3ce1-4f81-ad01-440d6b07c398 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:55.117231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db2986-3ce1-4f81-ad01-440d6b07c398 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:55.117676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db2986-3ce1-4f81-ad01-440d6b07c398 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:55.118169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db2986-3ce1-4f81-ad01-440d6b07c398 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:55.120367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f0db2986-3ce1-4f81-ad01-440d6b07c398 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:55.145126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 848/1712] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:29:54 2022] DELETE /identity/v3/users/cc04ad9577ae42b786b3a3c8da3ebb95 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:55.146710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa4c76-8bfa-4052-885c-31a2170385b2 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:55.147012 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa4c76-8bfa-4052-885c-31a2170385b2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:55.147289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa4c76-8bfa-4052-885c-31a2170385b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:55.155534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db2986-3ce1-4f81-ad01-440d6b07c398 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:55.158028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db2986-3ce1-4f81-ad01-440d6b07c398 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:55.180111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 865/1713] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:29:55.194235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-81aa4c76-8bfa-4052-885c-31a2170385b2 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:55.195816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-81aa4c76-8bfa-4052-885c-31a2170385b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:55.334398 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 849/1714] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:55.360971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db2986-3ce1-4f81-ad01-440d6b07c398 service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:55.362788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f0db2986-3ce1-4f81-ad01-440d6b07c398 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:55.386629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db2986-3ce1-4f81-ad01-440d6b07c398 service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:55.547943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:55.591014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b089d3f-13c2-4ee3-af21-387e6fc31b5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:55.592815 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b089d3f-13c2-4ee3-af21-387e6fc31b5a service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:55.594291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b089d3f-13c2-4ee3-af21-387e6fc31b5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:55.594291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b089d3f-13c2-4ee3-af21-387e6fc31b5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:55.635108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b089d3f-13c2-4ee3-af21-387e6fc31b5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:55.637578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b089d3f-13c2-4ee3-af21-387e6fc31b5a service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:55.666552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 866/1715] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:56.059726 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd5188-6b8c-44c2-9e55-a1550a42656b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:56.060244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd5188-6b8c-44c2-9e55-a1550a42656b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:56.060404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd5188-6b8c-44c2-9e55-a1550a42656b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:56.111051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7abd5188-6b8c-44c2-9e55-a1550a42656b None None] MFA Rules not processed for user `f4c978e96d3248d59120c287ecc75e87`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:56.113527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7abd5188-6b8c-44c2-9e55-a1550a42656b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:56.358650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 850/1716] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:58.033128 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b089d3f-13c2-4ee3-af21-387e6fc31b5a service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:58.035113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7b089d3f-13c2-4ee3-af21-387e6fc31b5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:58.074484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b089d3f-13c2-4ee3-af21-387e6fc31b5a service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:58.112226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e53147-6250-4232-8158-f2302f2fe8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:58.115214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e53147-6250-4232-8158-f2302f2fe8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:58.116448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e53147-6250-4232-8158-f2302f2fe8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:58.118220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e53147-6250-4232-8158-f2302f2fe8df admin admin] PATH_INFO: `/v3/projects/31fb04878484436380a4064e97ed9763` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:58.126095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e53147-6250-4232-8158-f2302f2fe8df admin admin] RBAC: Authorizing `identity:delete_project(project_id=31fb04878484436380a4064e97ed9763)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:58.131852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e53147-6250-4232-8158-f2302f2fe8df admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:58.308915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c4e53147-6250-4232-8158-f2302f2fe8df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31fb04878484436380a4064e97ed9763', 'request_id': 'req-c4e53147-6250-4232-8158-f2302f2fe8df'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:58.328072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c4e53147-6250-4232-8158-f2302f2fe8df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31fb04878484436380a4064e97ed9763', 'request_id': 'req-c4e53147-6250-4232-8158-f2302f2fe8df'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:58.346089 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c4e53147-6250-4232-8158-f2302f2fe8df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31fb04878484436380a4064e97ed9763', 'request_id': 'req-c4e53147-6250-4232-8158-f2302f2fe8df'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:58.374945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c4e53147-6250-4232-8158-f2302f2fe8df admin admin] The token cache is being invalidate because project 31fb04878484436380a4064e97ed9763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:29:58.375836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c4e53147-6250-4232-8158-f2302f2fe8df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:58.380515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c4e53147-6250-4232-8158-f2302f2fe8df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:58.385504 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 867/1717] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:29:58 2022] DELETE /identity/v3/projects/31fb04878484436380a4064e97ed9763 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:29:58.389285 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd5188-6b8c-44c2-9e55-a1550a42656b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:58.415367 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd5188-6b8c-44c2-9e55-a1550a42656b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:58.554158 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:58.644539 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a57571-1cc0-4c9c-97ef-34568fb1234b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:58.647762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a57571-1cc0-4c9c-97ef-34568fb1234b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:58.657511 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a57571-1cc0-4c9c-97ef-34568fb1234b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:58.658062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a57571-1cc0-4c9c-97ef-34568fb1234b admin admin] PATH_INFO: `/v3/users/a841ac78e41e4611bd0f18bb818c9d18` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:58.659797 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a57571-1cc0-4c9c-97ef-34568fb1234b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a841ac78e41e4611bd0f18bb818c9d18)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:58.662044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a57571-1cc0-4c9c-97ef-34568fb1234b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:58.766972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-30a57571-1cc0-4c9c-97ef-34568fb1234b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a841ac78e41e4611bd0f18bb818c9d18'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:58.783703 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-30a57571-1cc0-4c9c-97ef-34568fb1234b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a841ac78e41e4611bd0f18bb818c9d18'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:58.829501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-30a57571-1cc0-4c9c-97ef-34568fb1234b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a841ac78e41e4611bd0f18bb818c9d18'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:58.830766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-30a57571-1cc0-4c9c-97ef-34568fb1234b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a841ac78e41e4611bd0f18bb818c9d18'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:58.831496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-30a57571-1cc0-4c9c-97ef-34568fb1234b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a841ac78e41e4611bd0f18bb818c9d18'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:29:58.863069 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 851/1718] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:29:58 2022] DELETE /identity/v3/users/a841ac78e41e4611bd0f18bb818c9d18 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:29:58.917587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9865d22-60ae-41f6-9617-0c9483475bba None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:58.918102 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9865d22-60ae-41f6-9617-0c9483475bba None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:58.918565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9865d22-60ae-41f6-9617-0c9483475bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:59.050211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c9865d22-60ae-41f6-9617-0c9483475bba None None] MFA Rules not processed for user `065c988c57fa47e9b58af2914ff6306d`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:59.055343 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c9865d22-60ae-41f6-9617-0c9483475bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:59.397244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76919db-3fbf-412d-aa65-c19299303cdd None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:59.397774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76919db-3fbf-412d-aa65-c19299303cdd None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:59.397774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76919db-3fbf-412d-aa65-c19299303cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:59.399592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 868/1719] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:29:59.456423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c76919db-3fbf-412d-aa65-c19299303cdd None None] MFA Rules not processed for user `065c988c57fa47e9b58af2914ff6306d`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:59.465696 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c76919db-3fbf-412d-aa65-c19299303cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:59.500851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 852/1720] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:29:59.516676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9865d22-60ae-41f6-9617-0c9483475bba None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:29:59.518727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c9865d22-60ae-41f6-9617-0c9483475bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:59.676803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:29:59.719376 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371aa44-826e-45d9-9bab-7215f0246e7a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:59.719659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371aa44-826e-45d9-9bab-7215f0246e7a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:59.720118 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371aa44-826e-45d9-9bab-7215f0246e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:59.731456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9865d22-60ae-41f6-9617-0c9483475bba None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:29:59.738736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc01326-bd1a-4fd4-9b7a-e438c7d7bfb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:29:59.739402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc01326-bd1a-4fd4-9b7a-e438c7d7bfb8 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:29:59.739683 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc01326-bd1a-4fd4-9b7a-e438c7d7bfb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:29:59.739952 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc01326-bd1a-4fd4-9b7a-e438c7d7bfb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:29:59.762105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc01326-bd1a-4fd4-9b7a-e438c7d7bfb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:29:59.771736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc01326-bd1a-4fd4-9b7a-e438c7d7bfb8 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:29:59.786753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b371aa44-826e-45d9-9bab-7215f0246e7a None None] MFA Rules not processed for user `32df696da09746748786b94aa8b6721d`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:29:59.787983 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 869/1721] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:29:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:29:59.788908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b371aa44-826e-45d9-9bab-7215f0246e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:29:59.992217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 853/1722] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:29:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:00.432644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc01326-bd1a-4fd4-9b7a-e438c7d7bfb8 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:00.434946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9bc01326-bd1a-4fd4-9b7a-e438c7d7bfb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:00.455968 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc01326-bd1a-4fd4-9b7a-e438c7d7bfb8 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:00.472026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21da828-8e29-446d-b24b-f5ac92f48fd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:00.472876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21da828-8e29-446d-b24b-f5ac92f48fd7 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:00.473405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21da828-8e29-446d-b24b-f5ac92f48fd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:00.473793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21da828-8e29-446d-b24b-f5ac92f48fd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:00.497321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21da828-8e29-446d-b24b-f5ac92f48fd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:00.499023 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21da828-8e29-446d-b24b-f5ac92f48fd7 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:00.545752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 870/1723] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:00.931721 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371aa44-826e-45d9-9bab-7215f0246e7a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:00.935541 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b371aa44-826e-45d9-9bab-7215f0246e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:01.021059 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:01.160945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371aa44-826e-45d9-9bab-7215f0246e7a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:01.179250 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:01.179929 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:01.180246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:01.181169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c admin admin] PATH_INFO: `/v3/projects/46db4ed6e4a04c81ab9a7b338c523d43` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:01.184225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=46db4ed6e4a04c81ab9a7b338c523d43)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:01.186380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:01.209656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c3e5f-7749-4cb5-ba6f-89354032fefb None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:01.209957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c3e5f-7749-4cb5-ba6f-89354032fefb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:01.210318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c3e5f-7749-4cb5-ba6f-89354032fefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:01.271508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e65c3e5f-7749-4cb5-ba6f-89354032fefb None None] MFA Rules not processed for user `0d1dbe40cb994491bafc6ce1f947f4f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:01.272879 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e65c3e5f-7749-4cb5-ba6f-89354032fefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:01.303340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46db4ed6e4a04c81ab9a7b338c523d43', 'request_id': 'req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:01.318854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46db4ed6e4a04c81ab9a7b338c523d43', 'request_id': 'req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:01.328462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46db4ed6e4a04c81ab9a7b338c523d43', 'request_id': 'req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:01.353588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c admin admin] The token cache is being invalidate because project 46db4ed6e4a04c81ab9a7b338c523d43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:01.354034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:01.354966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c2713f81-0f51-4617-a1e6-e27f2eb06d9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:01.362895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 854/1724] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:00 2022] DELETE /identity/v3/projects/46db4ed6e4a04c81ab9a7b338c523d43 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:01.368778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d53ab0-6b90-431f-89f5-da99b8dbdad5 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:01.369069 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d53ab0-6b90-431f-89f5-da99b8dbdad5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:01.369341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d53ab0-6b90-431f-89f5-da99b8dbdad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:01.396379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c2d53ab0-6b90-431f-89f5-da99b8dbdad5 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:01.397786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c2d53ab0-6b90-431f-89f5-da99b8dbdad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:01.458678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 871/1725] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 11 05:30:01.472859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c3e5f-7749-4cb5-ba6f-89354032fefb None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:01.474884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e65c3e5f-7749-4cb5-ba6f-89354032fefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:01.507326 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 855/1726] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:01.517632 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7d982-8711-4bc8-a5f6-e0c9cb1ccc31 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:01.518772 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7d982-8711-4bc8-a5f6-e0c9cb1ccc31 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:01.519166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7d982-8711-4bc8-a5f6-e0c9cb1ccc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:01.546474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c3e5f-7749-4cb5-ba6f-89354032fefb None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:01.558769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-60b7d982-8711-4bc8-a5f6-e0c9cb1ccc31 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:01.561053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-60b7d982-8711-4bc8-a5f6-e0c9cb1ccc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:01.571705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 856/1727] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:01.575561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7d982-8711-4bc8-a5f6-e0c9cb1ccc31 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:01.622243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7d982-8711-4bc8-a5f6-e0c9cb1ccc31 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:01.644937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc05ba-f4c6-44fb-84a1-8943057db85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:01.647324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc05ba-f4c6-44fb-84a1-8943057db85f admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:01.647611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc05ba-f4c6-44fb-84a1-8943057db85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:01.647884 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc05ba-f4c6-44fb-84a1-8943057db85f admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:01.648607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc05ba-f4c6-44fb-84a1-8943057db85f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:30:01.648910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc05ba-f4c6-44fb-84a1-8943057db85f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:01.657892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc05ba-f4c6-44fb-84a1-8943057db85f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:01.683467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 857/1728] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:30:01.688543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc05ba-f4c6-44fb-84a1-8943057db85f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:01.722710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc05ba-f4c6-44fb-84a1-8943057db85f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:01.742554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be30f2d-7237-4c19-b058-de8499b81299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:01.744953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be30f2d-7237-4c19-b058-de8499b81299 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:01.746470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be30f2d-7237-4c19-b058-de8499b81299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:01.747041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be30f2d-7237-4c19-b058-de8499b81299 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:01.750057 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be30f2d-7237-4c19-b058-de8499b81299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:01.765844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be30f2d-7237-4c19-b058-de8499b81299 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:01.797647 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8856dd3-730e-448b-9fc8-806c97b7802f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:01.798348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8856dd3-730e-448b-9fc8-806c97b7802f service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:01.798733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8856dd3-730e-448b-9fc8-806c97b7802f service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:01.805834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8856dd3-730e-448b-9fc8-806c97b7802f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:01.856688 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 858/1729] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:01 2022] POST /identity/v3/projects => generated 367 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:01.860257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be30f2d-7237-4c19-b058-de8499b81299 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:01.984774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:02.016785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:02.026471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8856dd3-730e-448b-9fc8-806c97b7802f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:02.028675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8856dd3-730e-448b-9fc8-806c97b7802f service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:02.043605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 872/1730] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:30:02.076078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be30f2d-7237-4c19-b058-de8499b81299 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:02.087945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9fdc9-6934-48bc-b97b-3f4159e77274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:02.088626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9fdc9-6934-48bc-b97b-3f4159e77274 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:02.088914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9fdc9-6934-48bc-b97b-3f4159e77274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:02.089193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9fdc9-6934-48bc-b97b-3f4159e77274 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:02.091177 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9fdc9-6934-48bc-b97b-3f4159e77274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:02.097672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9fdc9-6934-48bc-b97b-3f4159e77274 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:02.147286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 859/1731] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:01 2022] POST /identity/v3/users => generated 284 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:02.149882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8856dd3-730e-448b-9fc8-806c97b7802f service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:02.175878 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8856dd3-730e-448b-9fc8-806c97b7802f service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:02.197641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89840f5b-2e34-4b33-af21-252f5b0cb8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:02.198363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89840f5b-2e34-4b33-af21-252f5b0cb8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:02.198733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89840f5b-2e34-4b33-af21-252f5b0cb8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:02.199063 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89840f5b-2e34-4b33-af21-252f5b0cb8e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:02.199845 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89840f5b-2e34-4b33-af21-252f5b0cb8e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:02.202090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89840f5b-2e34-4b33-af21-252f5b0cb8e3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:02.229293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 873/1732] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:02.234060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9fdc9-6934-48bc-b97b-3f4159e77274 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:02.255314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9fdc9-6934-48bc-b97b-3f4159e77274 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:02.272732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9f656-64d8-4986-9465-dec0d780fb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:02.274342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9f656-64d8-4986-9465-dec0d780fb4e admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:02.274630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9f656-64d8-4986-9465-dec0d780fb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:02.274901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9f656-64d8-4986-9465-dec0d780fb4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:02.280533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9f656-64d8-4986-9465-dec0d780fb4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:02.286762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9f656-64d8-4986-9465-dec0d780fb4e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:02.316911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 860/1733] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:02.321514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89840f5b-2e34-4b33-af21-252f5b0cb8e3 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:02.337974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89840f5b-2e34-4b33-af21-252f5b0cb8e3 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:02.361945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec5adb-ff5d-4d8d-8515-8f3620dcdb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:02.363208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec5adb-ff5d-4d8d-8515-8f3620dcdb80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:02.363591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec5adb-ff5d-4d8d-8515-8f3620dcdb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:02.363967 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec5adb-ff5d-4d8d-8515-8f3620dcdb80 admin admin] PATH_INFO: `/v3/projects/dd4adf2e5aed4ca3b7358832c3381d84/users/272f18415cc84ff193814c4a80041b35/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:02.393361 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec5adb-ff5d-4d8d-8515-8f3620dcdb80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4adf2e5aed4ca3b7358832c3381d84, user_id=272f18415cc84ff193814c4a80041b35, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:02.396697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec5adb-ff5d-4d8d-8515-8f3620dcdb80 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:02.432290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 874/1734] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:02 2022] PUT /identity/v3/projects/dd4adf2e5aed4ca3b7358832c3381d84/users/272f18415cc84ff193814c4a80041b35/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:02.443595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72361439-bf51-4dfe-95dd-d215f5daf5e6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:02.443878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72361439-bf51-4dfe-95dd-d215f5daf5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:02.444247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72361439-bf51-4dfe-95dd-d215f5daf5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:02.534646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-72361439-bf51-4dfe-95dd-d215f5daf5e6 None None] MFA Rules not processed for user `272f18415cc84ff193814c4a80041b35`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:02.536272 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-72361439-bf51-4dfe-95dd-d215f5daf5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:02.676358 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 861/1735] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:02.682691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec5adb-ff5d-4d8d-8515-8f3620dcdb80 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:02.732272 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282a55e-acd6-44fd-957e-b3462832b171 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:02.732567 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282a55e-acd6-44fd-957e-b3462832b171 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:02.732911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282a55e-acd6-44fd-957e-b3462832b171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:02.871921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec5adb-ff5d-4d8d-8515-8f3620dcdb80 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:02.872211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f282a55e-acd6-44fd-957e-b3462832b171 None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:02.873570 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f282a55e-acd6-44fd-957e-b3462832b171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:02.887234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bdf7f-650c-4d8c-ad32-aa4f13a7ff09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:02.887772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bdf7f-650c-4d8c-ad32-aa4f13a7ff09 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:02.888283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bdf7f-650c-4d8c-ad32-aa4f13a7ff09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:02.888775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bdf7f-650c-4d8c-ad32-aa4f13a7ff09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:02.889572 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2bdf7f-650c-4d8c-ad32-aa4f13a7ff09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:02.893346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2bdf7f-650c-4d8c-ad32-aa4f13a7ff09 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:02.962345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 875/1736] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:02 2022] POST /identity/v3/projects => generated 367 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:02.965971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bdf7f-650c-4d8c-ad32-aa4f13a7ff09 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:03.075827 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 862/1737] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 05:30:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:03.223611 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bdf7f-650c-4d8c-ad32-aa4f13a7ff09 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:03.248297 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8816a-4d37-4075-be4d-c4526fcde9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:03.249999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8816a-4d37-4075-be4d-c4526fcde9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:03.250745 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8816a-4d37-4075-be4d-c4526fcde9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:03.254093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8816a-4d37-4075-be4d-c4526fcde9b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:03.256356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8816a-4d37-4075-be4d-c4526fcde9b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:03.262141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8816a-4d37-4075-be4d-c4526fcde9b4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:03.362646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282a55e-acd6-44fd-957e-b3462832b171 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:03.364260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f282a55e-acd6-44fd-957e-b3462832b171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:03.370793 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:03.381714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 876/1738] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:02 2022] POST /identity/v3/users => generated 290 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:30:03.388512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8816a-4d37-4075-be4d-c4526fcde9b4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:03.438801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8816a-4d37-4075-be4d-c4526fcde9b4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:03.462799 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed428b-0922-4b93-b031-a91c5e957389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:03.463437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed428b-0922-4b93-b031-a91c5e957389 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:03.463718 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed428b-0922-4b93-b031-a91c5e957389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:03.464007 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed428b-0922-4b93-b031-a91c5e957389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:03.464677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed428b-0922-4b93-b031-a91c5e957389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:03.466393 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed428b-0922-4b93-b031-a91c5e957389 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:03.540272 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 877/1739] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:03.544440 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed428b-0922-4b93-b031-a91c5e957389 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:03.580203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed428b-0922-4b93-b031-a91c5e957389 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:03.594700 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771a2bf-0a94-49a1-a8d2-912d7d59c4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:03.597205 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771a2bf-0a94-49a1-a8d2-912d7d59c4e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:03.597540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771a2bf-0a94-49a1-a8d2-912d7d59c4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:03.597814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771a2bf-0a94-49a1-a8d2-912d7d59c4e3 admin admin] PATH_INFO: `/v3/projects/cd65f75551334f2ea90fe393bf086456/users/fbc25b083dc146d9b513f42355ff6a5f/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:03.663298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771a2bf-0a94-49a1-a8d2-912d7d59c4e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd65f75551334f2ea90fe393bf086456, user_id=fbc25b083dc146d9b513f42355ff6a5f, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:03.669289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771a2bf-0a94-49a1-a8d2-912d7d59c4e3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:03.709347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:03.729664 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 878/1740] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:03 2022] PUT /identity/v3/projects/cd65f75551334f2ea90fe393bf086456/users/fbc25b083dc146d9b513f42355ff6a5f/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:03.730995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771a2bf-0a94-49a1-a8d2-912d7d59c4e3 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:03.813311 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282a55e-acd6-44fd-957e-b3462832b171 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:03.932492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771a2bf-0a94-49a1-a8d2-912d7d59c4e3 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:03.945828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aeb1a-1961-4ea4-937d-4548e4947c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:03.946898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aeb1a-1961-4ea4-937d-4548e4947c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:03.947309 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aeb1a-1961-4ea4-937d-4548e4947c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:03.947747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aeb1a-1961-4ea4-937d-4548e4947c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:03.948575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0aeb1a-1961-4ea4-937d-4548e4947c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:03.950599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0aeb1a-1961-4ea4-937d-4548e4947c03 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:03.958938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0258d-1d3d-43cb-b9c3-ec6c5a13ee8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:03.960433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0258d-1d3d-43cb-b9c3-ec6c5a13ee8e service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:03.963906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0258d-1d3d-43cb-b9c3-ec6c5a13ee8e service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:03.964379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0258d-1d3d-43cb-b9c3-ec6c5a13ee8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:03.981960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 879/1741] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:03.983211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aeb1a-1961-4ea4-937d-4548e4947c03 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:04.010874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aeb1a-1961-4ea4-937d-4548e4947c03 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:04.022221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5aa0e8-9948-4a6d-829e-f6605ec41d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:04.022507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa0e8-9948-4a6d-829e-f6605ec41d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:04.022808 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa0e8-9948-4a6d-829e-f6605ec41d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:04.023890 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa0e8-9948-4a6d-829e-f6605ec41d7d admin admin] PATH_INFO: `/v3/projects/cd65f75551334f2ea90fe393bf086456/users/fbc25b083dc146d9b513f42355ff6a5f/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:04.031152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5aa0e8-9948-4a6d-829e-f6605ec41d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd65f75551334f2ea90fe393bf086456, user_id=fbc25b083dc146d9b513f42355ff6a5f, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:04.035154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5aa0e8-9948-4a6d-829e-f6605ec41d7d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:04.064439 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 880/1742] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:03 2022] PUT /identity/v3/projects/cd65f75551334f2ea90fe393bf086456/users/fbc25b083dc146d9b513f42355ff6a5f/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:04.079114 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22fb2c-a0c0-421b-bf23-82a3ec0e869c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:04.079216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22fb2c-a0c0-421b-bf23-82a3ec0e869c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:04.079488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22fb2c-a0c0-421b-bf23-82a3ec0e869c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:04.090374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0258d-1d3d-43cb-b9c3-ec6c5a13ee8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:04.092374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0258d-1d3d-43cb-b9c3-ec6c5a13ee8e service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:04.109961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 863/1743] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:30:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 748 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 05:30:04.144916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-db22fb2c-a0c0-421b-bf23-82a3ec0e869c None None] MFA Rules not processed for user `fbc25b083dc146d9b513f42355ff6a5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:04.148094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-db22fb2c-a0c0-421b-bf23-82a3ec0e869c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:04.356226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 881/1744] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:04.370248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0258d-1d3d-43cb-b9c3-ec6c5a13ee8e service swift] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:04.373734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d2a0258d-1d3d-43cb-b9c3-ec6c5a13ee8e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:04.503445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:04.521672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e00520-92d6-411c-9259-f6fe5d5fcec6 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:04.521979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e00520-92d6-411c-9259-f6fe5d5fcec6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:04.522433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e00520-92d6-411c-9259-f6fe5d5fcec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:04.580182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d0e00520-92d6-411c-9259-f6fe5d5fcec6 None None] MFA Rules not processed for user `32df696da09746748786b94aa8b6721d`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:04.584298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d0e00520-92d6-411c-9259-f6fe5d5fcec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:04.589196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0258d-1d3d-43cb-b9c3-ec6c5a13ee8e service swift] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:04.667532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e7710-d207-437c-be7e-19da6a153259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:04.668851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e7710-d207-437c-be7e-19da6a153259 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:04.669725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e7710-d207-437c-be7e-19da6a153259 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:04.670217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e7710-d207-437c-be7e-19da6a153259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:04.688310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 882/1745] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:04.693195 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e7710-d207-437c-be7e-19da6a153259 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:04.696851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e7710-d207-437c-be7e-19da6a153259 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:04.714483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 864/1746] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:07.389481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555bb9a-7331-48c9-b300-a210d47af0d3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:07.390054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555bb9a-7331-48c9-b300-a210d47af0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:07.390531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555bb9a-7331-48c9-b300-a210d47af0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:07.445245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3555bb9a-7331-48c9-b300-a210d47af0d3 None None] MFA Rules not processed for user `0d1dbe40cb994491bafc6ce1f947f4f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:07.446481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3555bb9a-7331-48c9-b300-a210d47af0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:07.466925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:07.486037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e7710-d207-437c-be7e-19da6a153259 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:07.587246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 883/1747] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:07.646526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e7710-d207-437c-be7e-19da6a153259 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:07.660971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:07.661733 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:07.662110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:07.662486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] PATH_INFO: `/v3/users/272f18415cc84ff193814c4a80041b35` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:07.664352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] RBAC: Authorizing `identity:delete_user(user_id=272f18415cc84ff193814c4a80041b35)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:07.670494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:07.832395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '272f18415cc84ff193814c4a80041b35'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:07.861743 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '272f18415cc84ff193814c4a80041b35'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:07.916339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '272f18415cc84ff193814c4a80041b35'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:07.941079 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '272f18415cc84ff193814c4a80041b35'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:07.942158 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '272f18415cc84ff193814c4a80041b35'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:08.009860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 865/1748] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:07 2022] DELETE /identity/v3/users/272f18415cc84ff193814c4a80041b35 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:08.015369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8441f-978e-4f75-a88f-021188ec9a15 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:08.016439 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8441f-978e-4f75-a88f-021188ec9a15 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:08.016782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8441f-978e-4f75-a88f-021188ec9a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:08.079219 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4db8441f-978e-4f75-a88f-021188ec9a15 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:08.087498 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4db8441f-978e-4f75-a88f-021188ec9a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:08.321863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 884/1749] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:08.385618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:08.387581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:08.398034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:08.451074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50144f77-80d3-49c7-9679-4ca02f1f8a41 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:08.453132 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50144f77-80d3-49c7-9679-4ca02f1f8a41 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:08.453819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50144f77-80d3-49c7-9679-4ca02f1f8a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:08.510730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-50144f77-80d3-49c7-9679-4ca02f1f8a41 None None] MFA Rules not processed for user `0bf2a5de0e744505a1317ce9fa224643`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:08.512518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-50144f77-80d3-49c7-9679-4ca02f1f8a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:08.529289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 885/1750] 10.222.0.17 () {58 vars in 1015 bytes} [Sun Sep 11 05:30:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:08.532663 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203469f-e487-40f8-be19-4902168c976c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:08.541470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51fc23-f32f-4485-979c-de430a370f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:08.544090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51fc23-f32f-4485-979c-de430a370f7f service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:08.544389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51fc23-f32f-4485-979c-de430a370f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:08.544675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51fc23-f32f-4485-979c-de430a370f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:08.607602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51fc23-f32f-4485-979c-de430a370f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:08.610338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51fc23-f32f-4485-979c-de430a370f7f service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:08.650695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50144f77-80d3-49c7-9679-4ca02f1f8a41 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:08.652730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-50144f77-80d3-49c7-9679-4ca02f1f8a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:08.671514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 866/1751] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:08.679775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50144f77-80d3-49c7-9679-4ca02f1f8a41 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:08.700888 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a6bfa-cfb1-4c16-b922-6504e3d2318b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:08.706531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a6bfa-cfb1-4c16-b922-6504e3d2318b service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:08.708372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a6bfa-cfb1-4c16-b922-6504e3d2318b service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:08.710429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a6bfa-cfb1-4c16-b922-6504e3d2318b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:08.747216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a6bfa-cfb1-4c16-b922-6504e3d2318b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:08.755358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a6bfa-cfb1-4c16-b922-6504e3d2318b service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:08.850966 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 886/1752] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:11.570828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51fc23-f32f-4485-979c-de430a370f7f service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:11.579461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fd51fc23-f32f-4485-979c-de430a370f7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:11.609710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51fc23-f32f-4485-979c-de430a370f7f service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:11.620384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:11.623093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:11.625289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:11.627417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] PATH_INFO: `/v3/users/5b02a00dce074792bafeb433d0ab4aa1` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:11.633874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b02a00dce074792bafeb433d0ab4aa1)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:11.641157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:11.862181 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b02a00dce074792bafeb433d0ab4aa1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:11.919648 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b02a00dce074792bafeb433d0ab4aa1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:12.012798 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b02a00dce074792bafeb433d0ab4aa1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:12.014200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b02a00dce074792bafeb433d0ab4aa1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:12.015742 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b02a00dce074792bafeb433d0ab4aa1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:12.106278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 867/1753] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:11 2022] DELETE /identity/v3/users/5b02a00dce074792bafeb433d0ab4aa1 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:12.120468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efe616-136a-48a3-a4aa-8c5a36d80220 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:12.120468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efe616-136a-48a3-a4aa-8c5a36d80220 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:12.120468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efe616-136a-48a3-a4aa-8c5a36d80220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:12.149015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b1efe616-136a-48a3-a4aa-8c5a36d80220 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:12.150645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b1efe616-136a-48a3-a4aa-8c5a36d80220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:12.302772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 887/1754] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:12.499629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:12.501567 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:12.743780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a9704-a17f-4a91-a471-4b34a838253f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:12.753132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d62ec8-3078-4ae0-89ad-16a4ba0a541d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:12.754289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d62ec8-3078-4ae0-89ad-16a4ba0a541d service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:12.754791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d62ec8-3078-4ae0-89ad-16a4ba0a541d service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:12.755278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d62ec8-3078-4ae0-89ad-16a4ba0a541d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:12.779307 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d62ec8-3078-4ae0-89ad-16a4ba0a541d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:12.781493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d62ec8-3078-4ae0-89ad-16a4ba0a541d service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:12.811344 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 868/1755] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:13.527575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efe616-136a-48a3-a4aa-8c5a36d80220 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:13.529055 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b1efe616-136a-48a3-a4aa-8c5a36d80220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:13.551091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efe616-136a-48a3-a4aa-8c5a36d80220 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:13.586555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:13.587412 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:13.587846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:13.590809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] PATH_INFO: `/v3/projects/dd4adf2e5aed4ca3b7358832c3381d84` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:13.596674 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4adf2e5aed4ca3b7358832c3381d84)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:13.600585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:13.682090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:13.697368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4adf2e5aed4ca3b7358832c3381d84', 'request_id': 'req-16964d42-2fa2-41bc-8de1-dff49e278435'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:13.706145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4adf2e5aed4ca3b7358832c3381d84', 'request_id': 'req-16964d42-2fa2-41bc-8de1-dff49e278435'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:13.722744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4adf2e5aed4ca3b7358832c3381d84', 'request_id': 'req-16964d42-2fa2-41bc-8de1-dff49e278435'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:13.745822 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] The token cache is being invalidate because project dd4adf2e5aed4ca3b7358832c3381d84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:13.746271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:13.747232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:13.750159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 888/1756] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:13 2022] DELETE /identity/v3/projects/dd4adf2e5aed4ca3b7358832c3381d84 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:13.753963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d62ec8-3078-4ae0-89ad-16a4ba0a541d service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:13.789074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d62ec8-3078-4ae0-89ad-16a4ba0a541d service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:13.836290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:13.926521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:13.928375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:13.928807 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:13.929247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] PATH_INFO: `/v3/users/fbc25b083dc146d9b513f42355ff6a5f` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:13.931219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbc25b083dc146d9b513f42355ff6a5f)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:13.947336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:14.056070 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbc25b083dc146d9b513f42355ff6a5f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.067869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbc25b083dc146d9b513f42355ff6a5f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.086030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbc25b083dc146d9b513f42355ff6a5f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.087009 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbc25b083dc146d9b513f42355ff6a5f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.088404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbc25b083dc146d9b513f42355ff6a5f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.175228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 869/1757] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:13 2022] DELETE /identity/v3/users/fbc25b083dc146d9b513f42355ff6a5f => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:14.180876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:14.183703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:14.205367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16964d42-2fa2-41bc-8de1-dff49e278435 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:14.289741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f1de5-980b-472b-9be9-2b91657fb059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:14.290449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f1de5-980b-472b-9be9-2b91657fb059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:14.290727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f1de5-980b-472b-9be9-2b91657fb059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:14.291043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f1de5-980b-472b-9be9-2b91657fb059 admin admin] PATH_INFO: `/v3/projects/2aec32d2b6714424a213b0d6cde5354a` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:14.293253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f1de5-980b-472b-9be9-2b91657fb059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aec32d2b6714424a213b0d6cde5354a)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:14.298341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f1de5-980b-472b-9be9-2b91657fb059 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:14.421998 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e29f1de5-980b-472b-9be9-2b91657fb059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aec32d2b6714424a213b0d6cde5354a', 'request_id': 'req-e29f1de5-980b-472b-9be9-2b91657fb059'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.433077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e29f1de5-980b-472b-9be9-2b91657fb059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aec32d2b6714424a213b0d6cde5354a', 'request_id': 'req-e29f1de5-980b-472b-9be9-2b91657fb059'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.445148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e29f1de5-980b-472b-9be9-2b91657fb059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aec32d2b6714424a213b0d6cde5354a', 'request_id': 'req-e29f1de5-980b-472b-9be9-2b91657fb059'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.472926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e29f1de5-980b-472b-9be9-2b91657fb059 admin admin] The token cache is being invalidate because project 2aec32d2b6714424a213b0d6cde5354a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:14.473781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e29f1de5-980b-472b-9be9-2b91657fb059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.474821 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e29f1de5-980b-472b-9be9-2b91657fb059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.477832 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 889/1758] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:14 2022] DELETE /identity/v3/projects/2aec32d2b6714424a213b0d6cde5354a => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:14.481812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:14.485131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:14.516325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0464aa-ce0a-405b-91e1-bf1473f80513 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:14.612387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334207bc-0b85-481b-87d5-ce5bcc7812ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:14.616269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334207bc-0b85-481b-87d5-ce5bcc7812ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:14.616963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334207bc-0b85-481b-87d5-ce5bcc7812ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:14.617505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334207bc-0b85-481b-87d5-ce5bcc7812ee admin admin] PATH_INFO: `/v3/users/dc1a61ce00d04103bfec53a16cd41cfd` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:14.620304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334207bc-0b85-481b-87d5-ce5bcc7812ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc1a61ce00d04103bfec53a16cd41cfd)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:14.627634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334207bc-0b85-481b-87d5-ce5bcc7812ee admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:14.701516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:14.706547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-334207bc-0b85-481b-87d5-ce5bcc7812ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc1a61ce00d04103bfec53a16cd41cfd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.722830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-334207bc-0b85-481b-87d5-ce5bcc7812ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc1a61ce00d04103bfec53a16cd41cfd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.768260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-334207bc-0b85-481b-87d5-ce5bcc7812ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc1a61ce00d04103bfec53a16cd41cfd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.771229 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-334207bc-0b85-481b-87d5-ce5bcc7812ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc1a61ce00d04103bfec53a16cd41cfd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.771883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-334207bc-0b85-481b-87d5-ce5bcc7812ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc1a61ce00d04103bfec53a16cd41cfd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:14.802793 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 870/1759] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:14 2022] DELETE /identity/v3/users/dc1a61ce00d04103bfec53a16cd41cfd => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:15.142706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ffebf-a781-4d53-9220-82ff9622589c None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:15.143641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ffebf-a781-4d53-9220-82ff9622589c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:15.144143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ffebf-a781-4d53-9220-82ff9622589c None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:15.148371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 890/1760] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 05:30:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:30:15.169968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc5977-8edb-4fd9-8683-bbb32ee378c4 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:15.171069 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc5977-8edb-4fd9-8683-bbb32ee378c4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:15.171761 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc5977-8edb-4fd9-8683-bbb32ee378c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:15.215359 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b7bc5977-8edb-4fd9-8683-bbb32ee378c4 None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:15.217212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b7bc5977-8edb-4fd9-8683-bbb32ee378c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:15.244659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ffebf-a781-4d53-9220-82ff9622589c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:15.246401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e82ffebf-a781-4d53-9220-82ff9622589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:15.349248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 871/1761] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 05:30:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:15.391271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:15.425829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ffebf-a781-4d53-9220-82ff9622589c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:15.432086 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:15.433133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:15.433590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:15.434043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] PATH_INFO: `/v3/projects/98fdca1edcf4442b86dbc19bc3612547` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:15.435432 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=98fdca1edcf4442b86dbc19bc3612547)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:15.439704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:15.482340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc5977-8edb-4fd9-8683-bbb32ee378c4 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:15.484694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b7bc5977-8edb-4fd9-8683-bbb32ee378c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:15.504274 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc5977-8edb-4fd9-8683-bbb32ee378c4 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:15.527068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca3500-1309-4d86-b787-579be1bbed5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:15.527835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca3500-1309-4d86-b787-579be1bbed5c service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:15.528250 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca3500-1309-4d86-b787-579be1bbed5c service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:15.528594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca3500-1309-4d86-b787-579be1bbed5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:15.542693 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca3500-1309-4d86-b787-579be1bbed5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:15.545503 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca3500-1309-4d86-b787-579be1bbed5c service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:15.560615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98fdca1edcf4442b86dbc19bc3612547', 'request_id': 'req-ce8431f0-9374-4e01-a277-3f37f68b3a0f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:15.561534 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 872/1762] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:15.571623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98fdca1edcf4442b86dbc19bc3612547', 'request_id': 'req-ce8431f0-9374-4e01-a277-3f37f68b3a0f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:15.583338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98fdca1edcf4442b86dbc19bc3612547', 'request_id': 'req-ce8431f0-9374-4e01-a277-3f37f68b3a0f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:15.604046 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] The token cache is being invalidate because project 98fdca1edcf4442b86dbc19bc3612547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:15.604851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:15.606475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:15.613182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 891/1763] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:15 2022] DELETE /identity/v3/projects/98fdca1edcf4442b86dbc19bc3612547 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:15.616426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33438b-ef04-479b-a6d3-51a8496a5e26 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:15.616938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33438b-ef04-479b-a6d3-51a8496a5e26 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:15.617063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33438b-ef04-479b-a6d3-51a8496a5e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:15.642987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:15.645263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-cc33438b-ef04-479b-a6d3-51a8496a5e26 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:15.646469 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cc33438b-ef04-479b-a6d3-51a8496a5e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:15.646791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:15.729971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 873/1764] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:15.737654 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23220719-d98e-4d41-bdd9-07f24cee45ef None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:15.737942 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23220719-d98e-4d41-bdd9-07f24cee45ef None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:15.738219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23220719-d98e-4d41-bdd9-07f24cee45ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:15.742041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8431f0-9374-4e01-a277-3f37f68b3a0f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:15.746823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edaed4b-20bd-4518-881c-7a83c794e39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:15.747554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaed4b-20bd-4518-881c-7a83c794e39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:15.747830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaed4b-20bd-4518-881c-7a83c794e39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:15.748138 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaed4b-20bd-4518-881c-7a83c794e39c admin admin] PATH_INFO: `/v3/projects/cd65f75551334f2ea90fe393bf086456` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:15.749535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edaed4b-20bd-4518-881c-7a83c794e39c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd65f75551334f2ea90fe393bf086456)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:15.751610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edaed4b-20bd-4518-881c-7a83c794e39c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:15.757449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-23220719-d98e-4d41-bdd9-07f24cee45ef None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:15.758938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-23220719-d98e-4d41-bdd9-07f24cee45ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:15.767578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 874/1765] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:15.771525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23220719-d98e-4d41-bdd9-07f24cee45ef None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:15.781038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23220719-d98e-4d41-bdd9-07f24cee45ef None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:15.786036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b985e3-60bc-42bd-b729-5820b90e3eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:15.786805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b985e3-60bc-42bd-b729-5820b90e3eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:15.787093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b985e3-60bc-42bd-b729-5820b90e3eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:15.787368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b985e3-60bc-42bd-b729-5820b90e3eb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:15.788038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b985e3-60bc-42bd-b729-5820b90e3eb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:30:15.788354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b985e3-60bc-42bd-b729-5820b90e3eb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:15.789968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b985e3-60bc-42bd-b729-5820b90e3eb1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:15.808256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 875/1766] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:30:15.809357 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b985e3-60bc-42bd-b729-5820b90e3eb1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:15.812658 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2edaed4b-20bd-4518-881c-7a83c794e39c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd65f75551334f2ea90fe393bf086456', 'request_id': 'req-2edaed4b-20bd-4518-881c-7a83c794e39c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:15.818315 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2edaed4b-20bd-4518-881c-7a83c794e39c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd65f75551334f2ea90fe393bf086456', 'request_id': 'req-2edaed4b-20bd-4518-881c-7a83c794e39c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:15.819950 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b985e3-60bc-42bd-b729-5820b90e3eb1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:15.825062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2edaed4b-20bd-4518-881c-7a83c794e39c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd65f75551334f2ea90fe393bf086456', 'request_id': 'req-2edaed4b-20bd-4518-881c-7a83c794e39c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:15.840219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:15.840428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2edaed4b-20bd-4518-881c-7a83c794e39c admin admin] The token cache is being invalidate because project cd65f75551334f2ea90fe393bf086456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:15.840788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2edaed4b-20bd-4518-881c-7a83c794e39c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:15.841436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-2edaed4b-20bd-4518-881c-7a83c794e39c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:15.845540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 892/1767] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:15 2022] DELETE /identity/v3/projects/cd65f75551334f2ea90fe393bf086456 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:15.849148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab4fff-1d01-4efd-9c42-62f356fe574e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:15.849265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab4fff-1d01-4efd-9c42-62f356fe574e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:15.849536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab4fff-1d01-4efd-9c42-62f356fe574e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:15.870451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-16ab4fff-1d01-4efd-9c42-62f356fe574e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:15.871755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-16ab4fff-1d01-4efd-9c42-62f356fe574e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:15.881130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:15.885751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24081c-09d3-469e-b669-cadfc1796c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:15.886701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24081c-09d3-469e-b669-cadfc1796c1e admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:15.887111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24081c-09d3-469e-b669-cadfc1796c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:15.887391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24081c-09d3-469e-b669-cadfc1796c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:15.888193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24081c-09d3-469e-b669-cadfc1796c1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:15.890462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24081c-09d3-469e-b669-cadfc1796c1e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:15.922343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 876/1768] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:15 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:15.926034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24081c-09d3-469e-b669-cadfc1796c1e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:15.927726 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1e24081c-09d3-469e-b669-cadfc1796c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:15.942736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 893/1769] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:15.950339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb98248-6860-47b9-8c56-2b005d6229c9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:15.950629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb98248-6860-47b9-8c56-2b005d6229c9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:15.950933 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb98248-6860-47b9-8c56-2b005d6229c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:15.974910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ebb98248-6860-47b9-8c56-2b005d6229c9 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:15.976168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ebb98248-6860-47b9-8c56-2b005d6229c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:15.997228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24081c-09d3-469e-b669-cadfc1796c1e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:16.002810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa52693-68fa-4bdd-b253-7a971a71732a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:16.003413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa52693-68fa-4bdd-b253-7a971a71732a admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:16.003690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa52693-68fa-4bdd-b253-7a971a71732a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:16.003960 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa52693-68fa-4bdd-b253-7a971a71732a admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:16.004614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa52693-68fa-4bdd-b253-7a971a71732a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:16.006531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa52693-68fa-4bdd-b253-7a971a71732a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:16.030116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 877/1770] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:15 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:30:16.034252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa52693-68fa-4bdd-b253-7a971a71732a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:16.044595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa52693-68fa-4bdd-b253-7a971a71732a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:16.045251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 894/1771] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:16.049779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb98248-6860-47b9-8c56-2b005d6229c9 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:16.049936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1e5c2-1a72-45f6-b18d-39477a91d421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:16.050446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1e5c2-1a72-45f6-b18d-39477a91d421 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:16.050723 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1e5c2-1a72-45f6-b18d-39477a91d421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:16.051003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1e5c2-1a72-45f6-b18d-39477a91d421 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:16.051622 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1e5c2-1a72-45f6-b18d-39477a91d421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:16.053569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1e5c2-1a72-45f6-b18d-39477a91d421 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:16.059271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb98248-6860-47b9-8c56-2b005d6229c9 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:16.064661 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ece202-e6db-43b8-8317-6c53b2d4d55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:16.065269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ece202-e6db-43b8-8317-6c53b2d4d55d admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:16.065552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ece202-e6db-43b8-8317-6c53b2d4d55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:16.065939 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ece202-e6db-43b8-8317-6c53b2d4d55d admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:16.066316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 878/1772] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:16.066601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ece202-e6db-43b8-8317-6c53b2d4d55d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:30:16.066913 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ece202-e6db-43b8-8317-6c53b2d4d55d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:16.068538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ece202-e6db-43b8-8317-6c53b2d4d55d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:16.070005 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1e5c2-1a72-45f6-b18d-39477a91d421 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:16.082536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 895/1773] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:30:16.083212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1e5c2-1a72-45f6-b18d-39477a91d421 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:16.086922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ece202-e6db-43b8-8317-6c53b2d4d55d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:16.089339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f1cb2-cdf3-4a1d-998e-6dbdcf751522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:16.090020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f1cb2-cdf3-4a1d-998e-6dbdcf751522 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:16.090363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f1cb2-cdf3-4a1d-998e-6dbdcf751522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:16.090706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f1cb2-cdf3-4a1d-998e-6dbdcf751522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:16.091423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f1cb2-cdf3-4a1d-998e-6dbdcf751522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:16.093554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f1cb2-cdf3-4a1d-998e-6dbdcf751522 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:16.099794 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ece202-e6db-43b8-8317-6c53b2d4d55d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:16.104716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 879/1774] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:16.105014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1194de-60c0-4077-929c-0dae53a78365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:16.108896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1194de-60c0-4077-929c-0dae53a78365 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:16.109188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1194de-60c0-4077-929c-0dae53a78365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:16.109414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f1cb2-cdf3-4a1d-998e-6dbdcf751522 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:16.109520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1194de-60c0-4077-929c-0dae53a78365 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:16.110190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1194de-60c0-4077-929c-0dae53a78365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:16.112182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1194de-60c0-4077-929c-0dae53a78365 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:16.122911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f1cb2-cdf3-4a1d-998e-6dbdcf751522 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:16.128577 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da467e-7b53-4146-a989-66f0754c13a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:16.129315 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da467e-7b53-4146-a989-66f0754c13a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:16.129661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da467e-7b53-4146-a989-66f0754c13a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:16.130070 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da467e-7b53-4146-a989-66f0754c13a2 admin admin] PATH_INFO: `/v3/projects/f4c47a2924b34a1db2925fcdbd8aaac2/users/fe9546aaab964048a51005e68e6a8576/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:16.147429 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da467e-7b53-4146-a989-66f0754c13a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c47a2924b34a1db2925fcdbd8aaac2, user_id=fe9546aaab964048a51005e68e6a8576, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:16.149337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 896/1775] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:16 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:16.150405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da467e-7b53-4146-a989-66f0754c13a2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:16.153099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1194de-60c0-4077-929c-0dae53a78365 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:16.165340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 880/1776] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:16 2022] PUT /identity/v3/projects/f4c47a2924b34a1db2925fcdbd8aaac2/users/fe9546aaab964048a51005e68e6a8576/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:16.185837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e705a88-b6e8-472d-ba54-f16302c5713d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:16.186131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e705a88-b6e8-472d-ba54-f16302c5713d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:16.187768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e705a88-b6e8-472d-ba54-f16302c5713d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:16.221205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3e705a88-b6e8-472d-ba54-f16302c5713d None None] MFA Rules not processed for user `fe9546aaab964048a51005e68e6a8576`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:16.222459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3e705a88-b6e8-472d-ba54-f16302c5713d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:16.250781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1194de-60c0-4077-929c-0dae53a78365 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:16.289997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:16.309655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:16.406277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 881/1777] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:16.411989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e705a88-b6e8-472d-ba54-f16302c5713d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:16.435692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107159d6-e0b4-4029-a883-f1b680fb109a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:16.436705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107159d6-e0b4-4029-a883-f1b680fb109a admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:16.436996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107159d6-e0b4-4029-a883-f1b680fb109a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:16.437403 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107159d6-e0b4-4029-a883-f1b680fb109a admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:16.438338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107159d6-e0b4-4029-a883-f1b680fb109a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:16.444399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107159d6-e0b4-4029-a883-f1b680fb109a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:16.510904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 897/1778] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:16 2022] POST /identity/v3/users => generated 283 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:16.515625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107159d6-e0b4-4029-a883-f1b680fb109a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:16.551151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107159d6-e0b4-4029-a883-f1b680fb109a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:16.590109 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47d99b-c896-42a0-bb2d-2f64f7fa8d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:16.590735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d99b-c896-42a0-bb2d-2f64f7fa8d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:16.591009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d99b-c896-42a0-bb2d-2f64f7fa8d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:16.591278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d99b-c896-42a0-bb2d-2f64f7fa8d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:16.592048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47d99b-c896-42a0-bb2d-2f64f7fa8d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:16.595482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47d99b-c896-42a0-bb2d-2f64f7fa8d1f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:16.618738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e705a88-b6e8-472d-ba54-f16302c5713d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:16.622289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 898/1779] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:16.629954 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47d99b-c896-42a0-bb2d-2f64f7fa8d1f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:16.635815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caadb4af-7557-45a7-b7e9-35466bbd4cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:16.640750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caadb4af-7557-45a7-b7e9-35466bbd4cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:16.641262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caadb4af-7557-45a7-b7e9-35466bbd4cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:16.641826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caadb4af-7557-45a7-b7e9-35466bbd4cc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:16.642478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caadb4af-7557-45a7-b7e9-35466bbd4cc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:16.644491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caadb4af-7557-45a7-b7e9-35466bbd4cc7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:16.658100 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47d99b-c896-42a0-bb2d-2f64f7fa8d1f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:16.668325 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ab1cd-e87d-4d7e-b97c-ddb53130f9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:16.668956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ab1cd-e87d-4d7e-b97c-ddb53130f9bc admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:16.669245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ab1cd-e87d-4d7e-b97c-ddb53130f9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:16.670480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ab1cd-e87d-4d7e-b97c-ddb53130f9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:16.671730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ab1cd-e87d-4d7e-b97c-ddb53130f9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:16.674725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ab1cd-e87d-4d7e-b97c-ddb53130f9bc admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:16.715029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 899/1780] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:16.723728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ab1cd-e87d-4d7e-b97c-ddb53130f9bc admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:16.726566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 882/1781] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:16 2022] POST /identity/v3/projects => generated 367 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:16.731833 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caadb4af-7557-45a7-b7e9-35466bbd4cc7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:16.749397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ab1cd-e87d-4d7e-b97c-ddb53130f9bc admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:16.860127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf21884-3738-43e4-8625-30a7b7a27f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:16.862255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf21884-3738-43e4-8625-30a7b7a27f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:16.862694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf21884-3738-43e4-8625-30a7b7a27f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:16.863116 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf21884-3738-43e4-8625-30a7b7a27f71 admin admin] PATH_INFO: `/v3/projects/4961d5916a8d4dc3863c2e243bfffdef/users/2c2da2e89c1f467ba65389646e9dc8f8/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:16.899881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf21884-3738-43e4-8625-30a7b7a27f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4961d5916a8d4dc3863c2e243bfffdef, user_id=2c2da2e89c1f467ba65389646e9dc8f8, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:16.903881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf21884-3738-43e4-8625-30a7b7a27f71 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:16.958926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 900/1782] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:16 2022] PUT /identity/v3/projects/4961d5916a8d4dc3863c2e243bfffdef/users/2c2da2e89c1f467ba65389646e9dc8f8/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:16.970967 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef603e2-0eb8-4d9a-a0fa-505e82af09cb None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:16.971646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef603e2-0eb8-4d9a-a0fa-505e82af09cb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:16.972039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef603e2-0eb8-4d9a-a0fa-505e82af09cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:16.977126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caadb4af-7557-45a7-b7e9-35466bbd4cc7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:17.039757 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0ef603e2-0eb8-4d9a-a0fa-505e82af09cb None None] MFA Rules not processed for user `2c2da2e89c1f467ba65389646e9dc8f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:17.041379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0ef603e2-0eb8-4d9a-a0fa-505e82af09cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:17.064908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60303e08-5325-4cd5-873a-3fef1abaf6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:17.065526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60303e08-5325-4cd5-873a-3fef1abaf6ed admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.065805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60303e08-5325-4cd5-873a-3fef1abaf6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.066076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60303e08-5325-4cd5-873a-3fef1abaf6ed admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.066766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60303e08-5325-4cd5-873a-3fef1abaf6ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:17.068832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60303e08-5325-4cd5-873a-3fef1abaf6ed admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:17.090516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:17.101841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 883/1783] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:16 2022] POST /identity/v3/users => generated 290 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:17.106533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60303e08-5325-4cd5-873a-3fef1abaf6ed admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:17.118957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60303e08-5325-4cd5-873a-3fef1abaf6ed admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:17.141153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac13fd9-fd3a-49a0-bc09-f3c3ea0c20e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:17.141551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 901/1784] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:17.142304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac13fd9-fd3a-49a0-bc09-f3c3ea0c20e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.142593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac13fd9-fd3a-49a0-bc09-f3c3ea0c20e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.142731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef603e2-0eb8-4d9a-a0fa-505e82af09cb None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:17.142914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac13fd9-fd3a-49a0-bc09-f3c3ea0c20e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.143530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac13fd9-fd3a-49a0-bc09-f3c3ea0c20e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:17.145220 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac13fd9-fd3a-49a0-bc09-f3c3ea0c20e2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:17.156194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef603e2-0eb8-4d9a-a0fa-505e82af09cb None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:17.157544 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 884/1785] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:17.161466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac13fd9-fd3a-49a0-bc09-f3c3ea0c20e2 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:17.164755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8f94d-f91d-419d-b1d6-6091ae8a98a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:17.165405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8f94d-f91d-419d-b1d6-6091ae8a98a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.165749 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8f94d-f91d-419d-b1d6-6091ae8a98a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.166175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8f94d-f91d-419d-b1d6-6091ae8a98a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.166813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8f94d-f91d-419d-b1d6-6091ae8a98a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:17.168772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8f94d-f91d-419d-b1d6-6091ae8a98a6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:17.172711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac13fd9-fd3a-49a0-bc09-f3c3ea0c20e2 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:17.174097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:17.178561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305709a-e8c1-46d6-901f-9ffe0a438fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:17.179727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305709a-e8c1-46d6-901f-9ffe0a438fad admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.180063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305709a-e8c1-46d6-901f-9ffe0a438fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.180332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305709a-e8c1-46d6-901f-9ffe0a438fad admin admin] PATH_INFO: `/v3/projects/17e12a8906264c3687329b4643cc85bc/users/9aaa71c5cffc4a99a5db98b2143569e7/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.199370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305709a-e8c1-46d6-901f-9ffe0a438fad admin admin] RBAC: Authorizing `identity:create_grant(project_id=17e12a8906264c3687329b4643cc85bc, user_id=9aaa71c5cffc4a99a5db98b2143569e7, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:17.207194 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305709a-e8c1-46d6-901f-9ffe0a438fad admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:17.225822 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 902/1786] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:17 2022] POST /identity/v3/projects => generated 367 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:17.230432 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8f94d-f91d-419d-b1d6-6091ae8a98a6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:17.231699 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 885/1787] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:17 2022] PUT /identity/v3/projects/17e12a8906264c3687329b4643cc85bc/users/9aaa71c5cffc4a99a5db98b2143569e7/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:17.236454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305709a-e8c1-46d6-901f-9ffe0a438fad admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:17.338072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8f94d-f91d-419d-b1d6-6091ae8a98a6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:17.343520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94154d7a-167d-4618-866e-ce2648116cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:17.344196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94154d7a-167d-4618-866e-ce2648116cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.344617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94154d7a-167d-4618-866e-ce2648116cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.344907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94154d7a-167d-4618-866e-ce2648116cb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.345550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94154d7a-167d-4618-866e-ce2648116cb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:17.347706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94154d7a-167d-4618-866e-ce2648116cb5 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:17.383551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 903/1788] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:17 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:30:17.394312 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94154d7a-167d-4618-866e-ce2648116cb5 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:17.395015 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305709a-e8c1-46d6-901f-9ffe0a438fad admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:17.401023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c69fd-4da8-4049-9b4c-ef8a6b5e4b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:17.401682 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c69fd-4da8-4049-9b4c-ef8a6b5e4b6e admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.401964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c69fd-4da8-4049-9b4c-ef8a6b5e4b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.402317 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c69fd-4da8-4049-9b4c-ef8a6b5e4b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.402947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c69fd-4da8-4049-9b4c-ef8a6b5e4b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:17.404652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c69fd-4da8-4049-9b4c-ef8a6b5e4b6e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:17.427365 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 886/1789] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:17.427594 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94154d7a-167d-4618-866e-ce2648116cb5 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:17.431701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c69fd-4da8-4049-9b4c-ef8a6b5e4b6e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:17.438243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ee0ee-97a5-49db-ab6c-b1824d5736c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:17.439767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ee0ee-97a5-49db-ab6c-b1824d5736c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.440169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ee0ee-97a5-49db-ab6c-b1824d5736c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.440548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ee0ee-97a5-49db-ab6c-b1824d5736c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.441317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ee0ee-97a5-49db-ab6c-b1824d5736c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:17.443181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ee0ee-97a5-49db-ab6c-b1824d5736c3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:17.451430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c69fd-4da8-4049-9b4c-ef8a6b5e4b6e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:17.466538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fe597-493a-4d84-843f-d6c59a778a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:17.467291 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fe597-493a-4d84-843f-d6c59a778a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.468769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fe597-493a-4d84-843f-d6c59a778a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.469237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fe597-493a-4d84-843f-d6c59a778a51 admin admin] PATH_INFO: `/v3/projects/17e12a8906264c3687329b4643cc85bc/users/9aaa71c5cffc4a99a5db98b2143569e7/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.471822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fe597-493a-4d84-843f-d6c59a778a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17e12a8906264c3687329b4643cc85bc, user_id=9aaa71c5cffc4a99a5db98b2143569e7, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:17.475183 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fe597-493a-4d84-843f-d6c59a778a51 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:17.476145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 904/1790] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:17.477217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ee0ee-97a5-49db-ab6c-b1824d5736c3 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:17.490628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ee0ee-97a5-49db-ab6c-b1824d5736c3 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:17.502218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 887/1791] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:17 2022] PUT /identity/v3/projects/17e12a8906264c3687329b4643cc85bc/users/9aaa71c5cffc4a99a5db98b2143569e7/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:17.504359 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66313937-072a-459a-84ce-bffbab7c8f0d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.504842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66313937-072a-459a-84ce-bffbab7c8f0d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.505249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66313937-072a-459a-84ce-bffbab7c8f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.542346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-66313937-072a-459a-84ce-bffbab7c8f0d None None] MFA Rules not processed for user `9aaa71c5cffc4a99a5db98b2143569e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:17.543795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-66313937-072a-459a-84ce-bffbab7c8f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:17.587313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fdac0-f2b1-4e62-995f-46082005b5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:17.587313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fdac0-f2b1-4e62-995f-46082005b5f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.587313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fdac0-f2b1-4e62-995f-46082005b5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.588419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fdac0-f2b1-4e62-995f-46082005b5f9 admin admin] PATH_INFO: `/v3/projects/201edf1868994849a77669c6629efad6/users/4ba1ea5234344dfa8c057fa055d522a7/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.618713 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fdac0-f2b1-4e62-995f-46082005b5f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=201edf1868994849a77669c6629efad6, user_id=4ba1ea5234344dfa8c057fa055d522a7, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:17.621635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fdac0-f2b1-4e62-995f-46082005b5f9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:17.642363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 888/1792] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:17.653896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 905/1793] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:17 2022] PUT /identity/v3/projects/201edf1868994849a77669c6629efad6/users/4ba1ea5234344dfa8c057fa055d522a7/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:17.658243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66313937-072a-459a-84ce-bffbab7c8f0d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:17.658678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fdac0-f2b1-4e62-995f-46082005b5f9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:17.659768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-66313937-072a-459a-84ce-bffbab7c8f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:17.739976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:17.778097 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66313937-072a-459a-84ce-bffbab7c8f0d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:17.781411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:17.818496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fdac0-f2b1-4e62-995f-46082005b5f9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:17.823651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72033d-8ffc-46ca-b47a-fa857af2e656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:17.824330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72033d-8ffc-46ca-b47a-fa857af2e656 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.824611 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72033d-8ffc-46ca-b47a-fa857af2e656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.824883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72033d-8ffc-46ca-b47a-fa857af2e656 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.825533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72033d-8ffc-46ca-b47a-fa857af2e656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:17.828492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72033d-8ffc-46ca-b47a-fa857af2e656 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:17.851190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 906/1794] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:17.852385 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72033d-8ffc-46ca-b47a-fa857af2e656 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:17.866169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72033d-8ffc-46ca-b47a-fa857af2e656 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:17.872587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f27838-a79d-43fc-805a-5ead8a1305a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:17.873281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f27838-a79d-43fc-805a-5ead8a1305a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.873562 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f27838-a79d-43fc-805a-5ead8a1305a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.873891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f27838-a79d-43fc-805a-5ead8a1305a8 admin admin] PATH_INFO: `/v3/projects/201edf1868994849a77669c6629efad6/users/4ba1ea5234344dfa8c057fa055d522a7/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.876541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f27838-a79d-43fc-805a-5ead8a1305a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=201edf1868994849a77669c6629efad6, user_id=4ba1ea5234344dfa8c057fa055d522a7, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:17.879685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f27838-a79d-43fc-805a-5ead8a1305a8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:17.903720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 907/1795] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:17 2022] PUT /identity/v3/projects/201edf1868994849a77669c6629efad6/users/4ba1ea5234344dfa8c057fa055d522a7/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:17.904484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5980101-d450-4efd-8984-0e7084bf3610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:17.904484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5980101-d450-4efd-8984-0e7084bf3610 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.904484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5980101-d450-4efd-8984-0e7084bf3610 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.904484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5980101-d450-4efd-8984-0e7084bf3610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.907957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d58d7-969c-4bb4-b717-6c0904ba767f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:17.908277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d58d7-969c-4bb4-b717-6c0904ba767f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:17.908585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d58d7-969c-4bb4-b717-6c0904ba767f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:17.947892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-639d58d7-969c-4bb4-b717-6c0904ba767f None None] MFA Rules not processed for user `4ba1ea5234344dfa8c057fa055d522a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:17.949603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-639d58d7-969c-4bb4-b717-6c0904ba767f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:18.088246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5980101-d450-4efd-8984-0e7084bf3610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:18.090087 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5980101-d450-4efd-8984-0e7084bf3610 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:18.103272 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 889/1796] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:30:18.119079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 908/1797] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:18.150948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5980101-d450-4efd-8984-0e7084bf3610 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:18.152567 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d5980101-d450-4efd-8984-0e7084bf3610 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:18.179902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5980101-d450-4efd-8984-0e7084bf3610 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:18.224923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d58d7-969c-4bb4-b717-6c0904ba767f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:18.278376 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c5490-9197-4847-a24a-91c04f13ea7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:18.279024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c5490-9197-4847-a24a-91c04f13ea7d service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:18.279308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c5490-9197-4847-a24a-91c04f13ea7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:18.279582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c5490-9197-4847-a24a-91c04f13ea7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:18.291185 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c5490-9197-4847-a24a-91c04f13ea7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:18.292865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c5490-9197-4847-a24a-91c04f13ea7d service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:18.322727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 890/1798] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:18.335310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d58d7-969c-4bb4-b717-6c0904ba767f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:18.340160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3beb9-b2c1-4b9b-8e9e-9a27ea68ae2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:18.340800 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3beb9-b2c1-4b9b-8e9e-9a27ea68ae2e service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:18.341076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3beb9-b2c1-4b9b-8e9e-9a27ea68ae2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:18.341351 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3beb9-b2c1-4b9b-8e9e-9a27ea68ae2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:18.413774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3beb9-b2c1-4b9b-8e9e-9a27ea68ae2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:18.415552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3beb9-b2c1-4b9b-8e9e-9a27ea68ae2e service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:18.424724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c5490-9197-4847-a24a-91c04f13ea7d service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:18.426616 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 909/1799] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:18.435339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c5490-9197-4847-a24a-91c04f13ea7d service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:18.439929 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d0e5f-e95d-41e8-b812-e2186d05f8ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:18.440585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d0e5f-e95d-41e8-b812-e2186d05f8ce service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:18.440863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d0e5f-e95d-41e8-b812-e2186d05f8ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:18.441130 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d0e5f-e95d-41e8-b812-e2186d05f8ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:18.492120 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:18.517976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d0e5f-e95d-41e8-b812-e2186d05f8ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:18.519539 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d0e5f-e95d-41e8-b812-e2186d05f8ce service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:18.559828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 891/1800] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:30:22.240677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3beb9-b2c1-4b9b-8e9e-9a27ea68ae2e service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:22.251120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:22.311965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3beb9-b2c1-4b9b-8e9e-9a27ea68ae2e service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:22.317364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:22.318001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:22.318281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:22.318556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] PATH_INFO: `/v3/users/fe9546aaab964048a51005e68e6a8576` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:22.319696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe9546aaab964048a51005e68e6a8576)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:22.321630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:22.376478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe9546aaab964048a51005e68e6a8576'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:22.377162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe9546aaab964048a51005e68e6a8576'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:22.385178 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe9546aaab964048a51005e68e6a8576'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:22.394229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe9546aaab964048a51005e68e6a8576'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:22.416346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe9546aaab964048a51005e68e6a8576'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:22.424695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 910/1801] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:22 2022] DELETE /identity/v3/users/fe9546aaab964048a51005e68e6a8576 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:22.430917 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29cad4-efa1-4afb-bc27-0e82cda9a40d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:22.431846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29cad4-efa1-4afb-bc27-0e82cda9a40d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:22.432373 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29cad4-efa1-4afb-bc27-0e82cda9a40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:22.457833 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-1e29cad4-efa1-4afb-bc27-0e82cda9a40d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:22.459064 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1e29cad4-efa1-4afb-bc27-0e82cda9a40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:22.557523 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 892/1802] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:22.567186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:22.568677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:22.633291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef89461-1cd0-4e63-8cdb-0d2705605f99 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:22.637406 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899d0ea-77be-4bd1-a687-86362d437efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:22.638213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899d0ea-77be-4bd1-a687-86362d437efa service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:22.638620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899d0ea-77be-4bd1-a687-86362d437efa service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:22.639031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899d0ea-77be-4bd1-a687-86362d437efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:22.652807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899d0ea-77be-4bd1-a687-86362d437efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:22.654248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899d0ea-77be-4bd1-a687-86362d437efa service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:22.665155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 911/1803] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:23.225966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29cad4-efa1-4afb-bc27-0e82cda9a40d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:23.227979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1e29cad4-efa1-4afb-bc27-0e82cda9a40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:23.240807 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29cad4-efa1-4afb-bc27-0e82cda9a40d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:23.245794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:23.246462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:23.246748 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:23.247023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] PATH_INFO: `/v3/projects/f4c47a2924b34a1db2925fcdbd8aaac2` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:23.248196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4c47a2924b34a1db2925fcdbd8aaac2)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:23.250491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:23.300443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4c47a2924b34a1db2925fcdbd8aaac2', 'request_id': 'req-97860cfc-aa0e-4af0-aadd-d52bfe81da59'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:23.307350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4c47a2924b34a1db2925fcdbd8aaac2', 'request_id': 'req-97860cfc-aa0e-4af0-aadd-d52bfe81da59'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:23.314141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4c47a2924b34a1db2925fcdbd8aaac2', 'request_id': 'req-97860cfc-aa0e-4af0-aadd-d52bfe81da59'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:23.324938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] The token cache is being invalidate because project f4c47a2924b34a1db2925fcdbd8aaac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:23.325169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:23.325827 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:23.328047 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 893/1804] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:23 2022] DELETE /identity/v3/projects/f4c47a2924b34a1db2925fcdbd8aaac2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:23.332117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899d0ea-77be-4bd1-a687-86362d437efa service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:23.341527 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899d0ea-77be-4bd1-a687-86362d437efa service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:23.370336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:23.406815 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b8e28-a9c7-4b6c-b123-696b42664d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:23.407468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b8e28-a9c7-4b6c-b123-696b42664d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:23.407750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b8e28-a9c7-4b6c-b123-696b42664d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:23.408048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b8e28-a9c7-4b6c-b123-696b42664d9d admin admin] PATH_INFO: `/v3/users/9aaa71c5cffc4a99a5db98b2143569e7` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:23.409220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b8e28-a9c7-4b6c-b123-696b42664d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aaa71c5cffc4a99a5db98b2143569e7)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:23.411379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b8e28-a9c7-4b6c-b123-696b42664d9d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:23.463679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c40b8e28-a9c7-4b6c-b123-696b42664d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aaa71c5cffc4a99a5db98b2143569e7'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:23.464366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c40b8e28-a9c7-4b6c-b123-696b42664d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aaa71c5cffc4a99a5db98b2143569e7'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:23.471978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c40b8e28-a9c7-4b6c-b123-696b42664d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aaa71c5cffc4a99a5db98b2143569e7'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:23.480344 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c40b8e28-a9c7-4b6c-b123-696b42664d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aaa71c5cffc4a99a5db98b2143569e7'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:23.492925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c40b8e28-a9c7-4b6c-b123-696b42664d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aaa71c5cffc4a99a5db98b2143569e7'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:23.499842 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 912/1805] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:23 2022] DELETE /identity/v3/users/9aaa71c5cffc4a99a5db98b2143569e7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:23.856984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:23.858996 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:23.884774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97860cfc-aa0e-4af0-aadd-d52bfe81da59 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:23.962388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:23.963614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:23.964494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:23.965299 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15 admin admin] PATH_INFO: `/v3/projects/17e12a8906264c3687329b4643cc85bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:23.969945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17e12a8906264c3687329b4643cc85bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:23.972315 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:23.980420 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:24.033746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17e12a8906264c3687329b4643cc85bc', 'request_id': 'req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:24.042123 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17e12a8906264c3687329b4643cc85bc', 'request_id': 'req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:24.050650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17e12a8906264c3687329b4643cc85bc', 'request_id': 'req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:24.063257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15 admin admin] The token cache is being invalidate because project 17e12a8906264c3687329b4643cc85bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:24.063563 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:24.068564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0bd534cd-5fb9-4b0c-9d55-c5280f064e15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:24.070665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 894/1806] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:23 2022] DELETE /identity/v3/projects/17e12a8906264c3687329b4643cc85bc => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:24.080555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4baf7-c25f-42da-8432-15c36a123e7d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:24.080906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4baf7-c25f-42da-8432-15c36a123e7d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:24.081192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4baf7-c25f-42da-8432-15c36a123e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:24.104755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-afd4baf7-c25f-42da-8432-15c36a123e7d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:24.105936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-afd4baf7-c25f-42da-8432-15c36a123e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:24.167512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 913/1807] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:24.175643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240a994-12b4-4297-af0a-ff47a84cbc5d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:24.176152 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240a994-12b4-4297-af0a-ff47a84cbc5d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:24.176529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240a994-12b4-4297-af0a-ff47a84cbc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:24.203776 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0240a994-12b4-4297-af0a-ff47a84cbc5d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:24.205126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0240a994-12b4-4297-af0a-ff47a84cbc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:24.214168 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 895/1808] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:24.219578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4baf7-c25f-42da-8432-15c36a123e7d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:24.241689 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4baf7-c25f-42da-8432-15c36a123e7d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:24.261517 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c89a7-c6f1-45fe-a56d-17623d12b5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:24.262222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c89a7-c6f1-45fe-a56d-17623d12b5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:24.262570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c89a7-c6f1-45fe-a56d-17623d12b5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:24.262850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c89a7-c6f1-45fe-a56d-17623d12b5f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:24.263519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c89a7-c6f1-45fe-a56d-17623d12b5f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:30:24.263824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c89a7-c6f1-45fe-a56d-17623d12b5f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:24.265932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c89a7-c6f1-45fe-a56d-17623d12b5f3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:24.287326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 914/1809] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:30:24.291277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240a994-12b4-4297-af0a-ff47a84cbc5d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:24.302239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240a994-12b4-4297-af0a-ff47a84cbc5d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:24.307659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ce23-3ef9-443a-ac1c-0cc759964821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:24.308478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ce23-3ef9-443a-ac1c-0cc759964821 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:24.308843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ce23-3ef9-443a-ac1c-0cc759964821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:24.309119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ce23-3ef9-443a-ac1c-0cc759964821 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:24.309907 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2ce23-3ef9-443a-ac1c-0cc759964821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:24.312501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2ce23-3ef9-443a-ac1c-0cc759964821 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:24.346919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 896/1810] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:24 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:24.351336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c89a7-c6f1-45fe-a56d-17623d12b5f3 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:24.362274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c89a7-c6f1-45fe-a56d-17623d12b5f3 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:24.398899 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:24.430072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4985ac-f8d3-4407-a31b-97fa1e26e4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:24.430685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4985ac-f8d3-4407-a31b-97fa1e26e4ee admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:24.430963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4985ac-f8d3-4407-a31b-97fa1e26e4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:24.431216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4985ac-f8d3-4407-a31b-97fa1e26e4ee admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:24.431852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4985ac-f8d3-4407-a31b-97fa1e26e4ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:24.433926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4985ac-f8d3-4407-a31b-97fa1e26e4ee admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:24.457841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 915/1811] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:24 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:24.462046 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ce23-3ef9-443a-ac1c-0cc759964821 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:24.473091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ce23-3ef9-443a-ac1c-0cc759964821 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:24.477659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26fdc1-088a-4e10-a170-e127758d0da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:24.478251 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26fdc1-088a-4e10-a170-e127758d0da3 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:24.478529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26fdc1-088a-4e10-a170-e127758d0da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:24.478799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26fdc1-088a-4e10-a170-e127758d0da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:24.479471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26fdc1-088a-4e10-a170-e127758d0da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:24.481118 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26fdc1-088a-4e10-a170-e127758d0da3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:24.491492 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 897/1812] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:24.495472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4985ac-f8d3-4407-a31b-97fa1e26e4ee admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:24.505575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4985ac-f8d3-4407-a31b-97fa1e26e4ee admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:24.510419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296d420-a8b1-4b05-83f9-d56f8d58e62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:24.511011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296d420-a8b1-4b05-83f9-d56f8d58e62e admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:24.511289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296d420-a8b1-4b05-83f9-d56f8d58e62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:24.511564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296d420-a8b1-4b05-83f9-d56f8d58e62e admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:24.512230 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296d420-a8b1-4b05-83f9-d56f8d58e62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:24.513806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296d420-a8b1-4b05-83f9-d56f8d58e62e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:24.524257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 916/1813] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:24.528336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26fdc1-088a-4e10-a170-e127758d0da3 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:24.539190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26fdc1-088a-4e10-a170-e127758d0da3 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:24.543725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931541c6-12b8-478a-b8a0-9a49caf5aea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:24.544402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931541c6-12b8-478a-b8a0-9a49caf5aea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:24.544697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931541c6-12b8-478a-b8a0-9a49caf5aea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:24.544985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931541c6-12b8-478a-b8a0-9a49caf5aea4 admin admin] PATH_INFO: `/v3/projects/de31030d2e5b4c2abe5b01262bc0b86a/users/9022d7f303324ccbb2cbd84e30b7fe47/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:24.561874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931541c6-12b8-478a-b8a0-9a49caf5aea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de31030d2e5b4c2abe5b01262bc0b86a, user_id=9022d7f303324ccbb2cbd84e30b7fe47, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:24.564629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931541c6-12b8-478a-b8a0-9a49caf5aea4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:24.578166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 898/1814] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:24 2022] PUT /identity/v3/projects/de31030d2e5b4c2abe5b01262bc0b86a/users/9022d7f303324ccbb2cbd84e30b7fe47/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:24.588531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec91082-ea9a-401c-8cd7-5145df64d7d3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:24.588816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec91082-ea9a-401c-8cd7-5145df64d7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:24.589091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec91082-ea9a-401c-8cd7-5145df64d7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:24.619286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3ec91082-ea9a-401c-8cd7-5145df64d7d3 None None] MFA Rules not processed for user `9022d7f303324ccbb2cbd84e30b7fe47`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:24.620556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3ec91082-ea9a-401c-8cd7-5145df64d7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:24.700841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 917/1815] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:24.706417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931541c6-12b8-478a-b8a0-9a49caf5aea4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:24.773467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931541c6-12b8-478a-b8a0-9a49caf5aea4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:24.778098 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5d310-5b0d-4d59-b95a-4264d3b7a30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:24.778702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5d310-5b0d-4d59-b95a-4264d3b7a30b admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:24.778985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5d310-5b0d-4d59-b95a-4264d3b7a30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:24.779258 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5d310-5b0d-4d59-b95a-4264d3b7a30b admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:24.780842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5d310-5b0d-4d59-b95a-4264d3b7a30b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:24.782763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5d310-5b0d-4d59-b95a-4264d3b7a30b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:24.813110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 899/1816] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:24 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:24.817618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec91082-ea9a-401c-8cd7-5145df64d7d3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:24.829274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec91082-ea9a-401c-8cd7-5145df64d7d3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:24.886401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152541db-9171-4ac4-8763-a8c36063357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:24.887017 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152541db-9171-4ac4-8763-a8c36063357f admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:24.887295 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152541db-9171-4ac4-8763-a8c36063357f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:24.887569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152541db-9171-4ac4-8763-a8c36063357f admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:24.888246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152541db-9171-4ac4-8763-a8c36063357f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:24.891009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152541db-9171-4ac4-8763-a8c36063357f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:24.914319 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 918/1817] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:24 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:24.918245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5d310-5b0d-4d59-b95a-4264d3b7a30b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:24.931178 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5d310-5b0d-4d59-b95a-4264d3b7a30b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:24.940649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d59fa0-66ae-4dbc-8c34-3d222fbe0728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:24.941289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d59fa0-66ae-4dbc-8c34-3d222fbe0728 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:24.941721 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d59fa0-66ae-4dbc-8c34-3d222fbe0728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:24.942165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d59fa0-66ae-4dbc-8c34-3d222fbe0728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:24.942879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d59fa0-66ae-4dbc-8c34-3d222fbe0728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:24.944825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d59fa0-66ae-4dbc-8c34-3d222fbe0728 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:24.960833 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 900/1818] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:24.964879 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152541db-9171-4ac4-8763-a8c36063357f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:24.980368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152541db-9171-4ac4-8763-a8c36063357f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:24.995197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb38010-71d5-409c-98d9-f1e58d48743c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:24.995919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb38010-71d5-409c-98d9-f1e58d48743c admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:24.996238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb38010-71d5-409c-98d9-f1e58d48743c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:24.996511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb38010-71d5-409c-98d9-f1e58d48743c admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:24.997327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb38010-71d5-409c-98d9-f1e58d48743c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:24.999953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb38010-71d5-409c-98d9-f1e58d48743c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:25.012610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 919/1819] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:25.018251 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d59fa0-66ae-4dbc-8c34-3d222fbe0728 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:25.028524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:25.042546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d59fa0-66ae-4dbc-8c34-3d222fbe0728 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:25.042546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef854a-df65-4da9-af55-d88f537cd720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:25.042546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef854a-df65-4da9-af55-d88f537cd720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:25.042546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef854a-df65-4da9-af55-d88f537cd720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:25.042986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef854a-df65-4da9-af55-d88f537cd720 admin admin] PATH_INFO: `/v3/projects/ecc183484b0d482d96f8697895a78eb9/users/bc9a92bae63e49dc950acf13d351570a/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:25.062200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef854a-df65-4da9-af55-d88f537cd720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc183484b0d482d96f8697895a78eb9, user_id=bc9a92bae63e49dc950acf13d351570a, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:25.065143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef854a-df65-4da9-af55-d88f537cd720 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:25.082340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 901/1820] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:25 2022] PUT /identity/v3/projects/ecc183484b0d482d96f8697895a78eb9/users/bc9a92bae63e49dc950acf13d351570a/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:25.093658 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7498c47-406f-47e0-a0d9-9db4eb24814b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:25.093940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7498c47-406f-47e0-a0d9-9db4eb24814b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:25.094220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7498c47-406f-47e0-a0d9-9db4eb24814b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:25.121562 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a7498c47-406f-47e0-a0d9-9db4eb24814b None None] MFA Rules not processed for user `bc9a92bae63e49dc950acf13d351570a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:25.122773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a7498c47-406f-47e0-a0d9-9db4eb24814b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:25.135198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:25.191193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 920/1821] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:25.199098 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef854a-df65-4da9-af55-d88f537cd720 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:25.276361 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef854a-df65-4da9-af55-d88f537cd720 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:25.281438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f9f8d-3fd8-4d66-8cfa-53e62449d279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:25.282049 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f9f8d-3fd8-4d66-8cfa-53e62449d279 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:25.282325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f9f8d-3fd8-4d66-8cfa-53e62449d279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:25.282598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f9f8d-3fd8-4d66-8cfa-53e62449d279 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:25.283225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f9f8d-3fd8-4d66-8cfa-53e62449d279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:25.285173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f9f8d-3fd8-4d66-8cfa-53e62449d279 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:25.317544 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 902/1822] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:25 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:25.321625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7498c47-406f-47e0-a0d9-9db4eb24814b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:25.337849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7498c47-406f-47e0-a0d9-9db4eb24814b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:25.423151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35395165-fc62-49ac-915e-344314adfe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:25.423836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35395165-fc62-49ac-915e-344314adfe35 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:25.424095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35395165-fc62-49ac-915e-344314adfe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:25.424271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35395165-fc62-49ac-915e-344314adfe35 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:25.424928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35395165-fc62-49ac-915e-344314adfe35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:25.426867 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35395165-fc62-49ac-915e-344314adfe35 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:25.448945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 921/1823] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:25 2022] POST /identity/v3/users => generated 298 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:25.452654 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f9f8d-3fd8-4d66-8cfa-53e62449d279 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:25.468761 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f9f8d-3fd8-4d66-8cfa-53e62449d279 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:25.473509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8b95c-0968-423b-9427-43949fc6e2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:25.474125 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8b95c-0968-423b-9427-43949fc6e2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:25.474409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8b95c-0968-423b-9427-43949fc6e2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:25.474685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8b95c-0968-423b-9427-43949fc6e2f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:25.475336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8b95c-0968-423b-9427-43949fc6e2f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:25.476976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8b95c-0968-423b-9427-43949fc6e2f6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:25.492031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 903/1824] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:25.493697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35395165-fc62-49ac-915e-344314adfe35 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:25.505619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35395165-fc62-49ac-915e-344314adfe35 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:25.511520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0470b03-cfce-41d8-8ab3-637f37365151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:25.512457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0470b03-cfce-41d8-8ab3-637f37365151 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:25.512753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0470b03-cfce-41d8-8ab3-637f37365151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:25.512987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0470b03-cfce-41d8-8ab3-637f37365151 admin admin] PATH_INFO: `/v3/projects/40f4eb6387014bcc9b3270cda947aeff/users/64a2534b615b41f58f57e4e0bd598c5d/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:25.548917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0470b03-cfce-41d8-8ab3-637f37365151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40f4eb6387014bcc9b3270cda947aeff, user_id=64a2534b615b41f58f57e4e0bd598c5d, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:25.552105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0470b03-cfce-41d8-8ab3-637f37365151 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:25.571638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 922/1825] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:25 2022] PUT /identity/v3/projects/40f4eb6387014bcc9b3270cda947aeff/users/64a2534b615b41f58f57e4e0bd598c5d/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:25.575218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8b95c-0968-423b-9427-43949fc6e2f6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:25.586363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8b95c-0968-423b-9427-43949fc6e2f6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:25.655787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3f786-8243-4028-8879-b040bfab6609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:25.656629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3f786-8243-4028-8879-b040bfab6609 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:25.656909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3f786-8243-4028-8879-b040bfab6609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:25.657183 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3f786-8243-4028-8879-b040bfab6609 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:25.657889 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3f786-8243-4028-8879-b040bfab6609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:25.659500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3f786-8243-4028-8879-b040bfab6609 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:25.676103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 904/1826] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:25.679974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0470b03-cfce-41d8-8ab3-637f37365151 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:25.690424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0470b03-cfce-41d8-8ab3-637f37365151 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:25.695048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bb4d2-f098-4f45-b090-bd393581b526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:25.695698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bb4d2-f098-4f45-b090-bd393581b526 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:25.695972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bb4d2-f098-4f45-b090-bd393581b526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:25.696273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bb4d2-f098-4f45-b090-bd393581b526 admin admin] PATH_INFO: `/v3/projects/40f4eb6387014bcc9b3270cda947aeff/users/64a2534b615b41f58f57e4e0bd598c5d/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:25.698147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906bb4d2-f098-4f45-b090-bd393581b526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40f4eb6387014bcc9b3270cda947aeff, user_id=64a2534b615b41f58f57e4e0bd598c5d, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:25.700787 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906bb4d2-f098-4f45-b090-bd393581b526 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:25.711108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 923/1827] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:25 2022] PUT /identity/v3/projects/40f4eb6387014bcc9b3270cda947aeff/users/64a2534b615b41f58f57e4e0bd598c5d/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:25.721920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d041f9-d56d-40cc-8d10-6e8e01779fbd None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:25.722203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d041f9-d56d-40cc-8d10-6e8e01779fbd None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:25.722475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d041f9-d56d-40cc-8d10-6e8e01779fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:25.751037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a5d041f9-d56d-40cc-8d10-6e8e01779fbd None None] MFA Rules not processed for user `64a2534b615b41f58f57e4e0bd598c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:25.752277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a5d041f9-d56d-40cc-8d10-6e8e01779fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:25.792279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:25.825208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 905/1828] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:25.836681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bb4d2-f098-4f45-b090-bd393581b526 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:25.838377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-906bb4d2-f098-4f45-b090-bd393581b526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:25.911844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bb4d2-f098-4f45-b090-bd393581b526 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:25.915494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0468b6d-7445-4723-8ef7-cd25c9ef975f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:25.916165 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0468b6d-7445-4723-8ef7-cd25c9ef975f service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:25.916462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0468b6d-7445-4723-8ef7-cd25c9ef975f service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:25.916734 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0468b6d-7445-4723-8ef7-cd25c9ef975f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:25.935985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:25.976045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0468b6d-7445-4723-8ef7-cd25c9ef975f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:25.977492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0468b6d-7445-4723-8ef7-cd25c9ef975f service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:25.987285 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 924/1829] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:29.439695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d041f9-d56d-40cc-8d10-6e8e01779fbd None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:29.445595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a5d041f9-d56d-40cc-8d10-6e8e01779fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:29.708787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d041f9-d56d-40cc-8d10-6e8e01779fbd None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:29.727932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:29.730169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:29.730474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:29.731528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] PATH_INFO: `/v3/users/2c2da2e89c1f467ba65389646e9dc8f8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:29.736583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c2da2e89c1f467ba65389646e9dc8f8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:29.742069 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:29.947034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c2da2e89c1f467ba65389646e9dc8f8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:29.980225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c2da2e89c1f467ba65389646e9dc8f8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:30.038006 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c2da2e89c1f467ba65389646e9dc8f8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:30.038999 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c2da2e89c1f467ba65389646e9dc8f8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:30.039790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c2da2e89c1f467ba65389646e9dc8f8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:30.069446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 906/1830] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:29 2022] DELETE /identity/v3/users/2c2da2e89c1f467ba65389646e9dc8f8 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:30.070853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93470e55-b290-4c34-8015-b244ba4109fb None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:30.071413 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93470e55-b290-4c34-8015-b244ba4109fb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:30.071413 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93470e55-b290-4c34-8015-b244ba4109fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:30.115902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:30.118192 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:30.125755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-93470e55-b290-4c34-8015-b244ba4109fb None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:30.127187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-93470e55-b290-4c34-8015-b244ba4109fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:30.268407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4896879-30f3-4aa7-ad7f-072c14f3fdc6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:30.280936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2c56a-c7b7-4dce-b2ce-1ee781385840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:30.281854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2c56a-c7b7-4dce-b2ce-1ee781385840 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:30.282423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2c56a-c7b7-4dce-b2ce-1ee781385840 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:30.282707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2c56a-c7b7-4dce-b2ce-1ee781385840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:30.287230 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 925/1831] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:30.288920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6ff2c56a-c7b7-4dce-b2ce-1ee781385840 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:30.309467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:30.431822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2c56a-c7b7-4dce-b2ce-1ee781385840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:30.434012 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2c56a-c7b7-4dce-b2ce-1ee781385840 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:30.463242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 907/1832] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:30.471072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93470e55-b290-4c34-8015-b244ba4109fb None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:30.473165 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-93470e55-b290-4c34-8015-b244ba4109fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:30.601017 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93470e55-b290-4c34-8015-b244ba4109fb None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:30.605932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaf3dc-1c97-4a3e-871a-50d3e5c5691e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:30.608403 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaf3dc-1c97-4a3e-871a-50d3e5c5691e service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:30.609081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaf3dc-1c97-4a3e-871a-50d3e5c5691e service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:30.609559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaf3dc-1c97-4a3e-871a-50d3e5c5691e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:30.626378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeaf3dc-1c97-4a3e-871a-50d3e5c5691e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:30.629147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeaf3dc-1c97-4a3e-871a-50d3e5c5691e service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:30.644153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 926/1833] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:30.901207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02958cc3-2941-404a-985c-85bea423c01f None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:30.901611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02958cc3-2941-404a-985c-85bea423c01f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:30.901870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02958cc3-2941-404a-985c-85bea423c01f None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:30.903495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 908/1834] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 05:30:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:30:31.013043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76299b56-48fc-4a03-84bb-8fbeae1cde3d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:31.013292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76299b56-48fc-4a03-84bb-8fbeae1cde3d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:31.013687 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76299b56-48fc-4a03-84bb-8fbeae1cde3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:31.046716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-76299b56-48fc-4a03-84bb-8fbeae1cde3d None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:31.049353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-76299b56-48fc-4a03-84bb-8fbeae1cde3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:31.257461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:31.288831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 927/1835] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 05:30:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:31.519735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02958cc3-2941-404a-985c-85bea423c01f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:31.525506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76299b56-48fc-4a03-84bb-8fbeae1cde3d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:31.528380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-76299b56-48fc-4a03-84bb-8fbeae1cde3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:31.554086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02958cc3-2941-404a-985c-85bea423c01f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:31.561579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76299b56-48fc-4a03-84bb-8fbeae1cde3d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:31.564233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5861b6-cf8e-4574-8fa0-698163e3a84d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:31.565095 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5861b6-cf8e-4574-8fa0-698163e3a84d service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:31.565582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5861b6-cf8e-4574-8fa0-698163e3a84d service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:31.566062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5861b6-cf8e-4574-8fa0-698163e3a84d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:31.568717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-da5861b6-cf8e-4574-8fa0-698163e3a84d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:31.585935 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b89c7-f128-4a0f-8065-7d5ea429339a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:31.586893 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b89c7-f128-4a0f-8065-7d5ea429339a service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:31.587341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b89c7-f128-4a0f-8065-7d5ea429339a service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:31.587820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b89c7-f128-4a0f-8065-7d5ea429339a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:31.610993 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b89c7-f128-4a0f-8065-7d5ea429339a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:31.613061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b89c7-f128-4a0f-8065-7d5ea429339a service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:31.638308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 928/1836] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:31.732756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5861b6-cf8e-4574-8fa0-698163e3a84d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:31.734275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5861b6-cf8e-4574-8fa0-698163e3a84d service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:31.748150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 909/1837] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:32.750977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b89c7-f128-4a0f-8065-7d5ea429339a service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:32.756706 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1e6b89c7-f128-4a0f-8065-7d5ea429339a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:32.759815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5861b6-cf8e-4574-8fa0-698163e3a84d service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:32.764183 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-da5861b6-cf8e-4574-8fa0-698163e3a84d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:32.784953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b89c7-f128-4a0f-8065-7d5ea429339a service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:32.795833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662e5af-35e0-4d2b-88be-b1eafb35d009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:32.797291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662e5af-35e0-4d2b-88be-b1eafb35d009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:32.797984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662e5af-35e0-4d2b-88be-b1eafb35d009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:32.798852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662e5af-35e0-4d2b-88be-b1eafb35d009 admin admin] PATH_INFO: `/v3/users/116ea262cc704290b7c787a0f39949b5` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:32.801118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662e5af-35e0-4d2b-88be-b1eafb35d009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=116ea262cc704290b7c787a0f39949b5)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:32.805457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662e5af-35e0-4d2b-88be-b1eafb35d009 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:32.812101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5861b6-cf8e-4574-8fa0-698163e3a84d service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:32.821305 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:32.822356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:32.822845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:32.823455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] PATH_INFO: `/v3/projects/4961d5916a8d4dc3863c2e243bfffdef` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:32.825065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4961d5916a8d4dc3863c2e243bfffdef)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:32.827658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:32.895608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6662e5af-35e0-4d2b-88be-b1eafb35d009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '116ea262cc704290b7c787a0f39949b5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:32.896598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6662e5af-35e0-4d2b-88be-b1eafb35d009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '116ea262cc704290b7c787a0f39949b5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:32.905730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4961d5916a8d4dc3863c2e243bfffdef', 'request_id': 'req-06d98925-542f-4f67-be9b-e0a8f8b72dc8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:32.907890 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6662e5af-35e0-4d2b-88be-b1eafb35d009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '116ea262cc704290b7c787a0f39949b5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:32.919137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6662e5af-35e0-4d2b-88be-b1eafb35d009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '116ea262cc704290b7c787a0f39949b5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:32.923608 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4961d5916a8d4dc3863c2e243bfffdef', 'request_id': 'req-06d98925-542f-4f67-be9b-e0a8f8b72dc8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:32.934196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4961d5916a8d4dc3863c2e243bfffdef', 'request_id': 'req-06d98925-542f-4f67-be9b-e0a8f8b72dc8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:32.939909 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6662e5af-35e0-4d2b-88be-b1eafb35d009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '116ea262cc704290b7c787a0f39949b5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:32.949381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] The token cache is being invalidate because project 4961d5916a8d4dc3863c2e243bfffdef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:32.949839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:32.950409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:32.950519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 929/1838] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:32 2022] DELETE /identity/v3/users/116ea262cc704290b7c787a0f39949b5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:32.954554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 910/1839] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:32 2022] DELETE /identity/v3/projects/4961d5916a8d4dc3863c2e243bfffdef => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:32.957402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938571d-c847-4d3e-9b9a-1334e1049cff None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:32.957698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938571d-c847-4d3e-9b9a-1334e1049cff None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:32.958594 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938571d-c847-4d3e-9b9a-1334e1049cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:32.958802 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:32.960561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:32.980368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2938571d-c847-4d3e-9b9a-1334e1049cff None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:32.984051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2938571d-c847-4d3e-9b9a-1334e1049cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:33.023216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:33.092832 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 930/1840] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:33.128161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938571d-c847-4d3e-9b9a-1334e1049cff None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:33.128294 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d98925-542f-4f67-be9b-e0a8f8b72dc8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:33.132422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2938571d-c847-4d3e-9b9a-1334e1049cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:33.149333 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:33.151232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:33.155526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:33.155921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] PATH_INFO: `/v3/users/4ba1ea5234344dfa8c057fa055d522a7` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:33.159906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ba1ea5234344dfa8c057fa055d522a7)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:33.163149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:33.283636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938571d-c847-4d3e-9b9a-1334e1049cff None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:33.288519 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ba1ea5234344dfa8c057fa055d522a7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:33.288697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fac6a-3cf8-4081-b722-41687289aee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:33.289199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fac6a-3cf8-4081-b722-41687289aee2 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:33.289586 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fac6a-3cf8-4081-b722-41687289aee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:33.289964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fac6a-3cf8-4081-b722-41687289aee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:33.300637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ba1ea5234344dfa8c057fa055d522a7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:33.304343 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fac6a-3cf8-4081-b722-41687289aee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:33.306490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fac6a-3cf8-4081-b722-41687289aee2 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:33.313337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:33.336652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 931/1841] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:33.339639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ba1ea5234344dfa8c057fa055d522a7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:33.340921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ba1ea5234344dfa8c057fa055d522a7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:33.343686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ba1ea5234344dfa8c057fa055d522a7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:33.378158 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 911/1842] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:32 2022] DELETE /identity/v3/users/4ba1ea5234344dfa8c057fa055d522a7 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:34.134724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fac6a-3cf8-4081-b722-41687289aee2 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:34.138716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4f7fac6a-3cf8-4081-b722-41687289aee2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:34.158694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fac6a-3cf8-4081-b722-41687289aee2 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:34.160526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:34.162502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:34.243465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03726a-8859-4e5e-98b4-59a59f50621e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:34.246769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:34.247676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:34.248227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:34.249156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] PATH_INFO: `/v3/projects/e65940f72e054523aa100c10e3824c37` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:34.249327 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d620a-3667-4e41-bfeb-224406578083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:34.250094 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d620a-3667-4e41-bfeb-224406578083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:34.250373 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d620a-3667-4e41-bfeb-224406578083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:34.250638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d620a-3667-4e41-bfeb-224406578083 admin admin] PATH_INFO: `/v3/projects/201edf1868994849a77669c6629efad6` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:34.250746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e65940f72e054523aa100c10e3824c37)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:34.252205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d620a-3667-4e41-bfeb-224406578083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=201edf1868994849a77669c6629efad6)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:34.253057 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:34.255254 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d620a-3667-4e41-bfeb-224406578083 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:34.306906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e65940f72e054523aa100c10e3824c37', 'request_id': 'req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.308207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:34.309972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0b9d620a-3667-4e41-bfeb-224406578083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201edf1868994849a77669c6629efad6', 'request_id': 'req-0b9d620a-3667-4e41-bfeb-224406578083'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.320884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e65940f72e054523aa100c10e3824c37', 'request_id': 'req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.325457 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0b9d620a-3667-4e41-bfeb-224406578083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201edf1868994849a77669c6629efad6', 'request_id': 'req-0b9d620a-3667-4e41-bfeb-224406578083'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.329749 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e65940f72e054523aa100c10e3824c37', 'request_id': 'req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.334159 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0b9d620a-3667-4e41-bfeb-224406578083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201edf1868994849a77669c6629efad6', 'request_id': 'req-0b9d620a-3667-4e41-bfeb-224406578083'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.344348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] The token cache is being invalidate because project e65940f72e054523aa100c10e3824c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:34.344787 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.345589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.350797 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 932/1843] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:34 2022] DELETE /identity/v3/projects/e65940f72e054523aa100c10e3824c37 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:34.350939 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0b9d620a-3667-4e41-bfeb-224406578083 admin admin] The token cache is being invalidate because project 201edf1868994849a77669c6629efad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:34.350939 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0b9d620a-3667-4e41-bfeb-224406578083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.350939 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0b9d620a-3667-4e41-bfeb-224406578083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.351814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:34.352682 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 912/1844] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:34 2022] DELETE /identity/v3/projects/201edf1868994849a77669c6629efad6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:34.356651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:34.359709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2a184-5abb-4a4f-9f9f-2b3a2369ec58 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:34.360025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2a184-5abb-4a4f-9f9f-2b3a2369ec58 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:34.360427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2a184-5abb-4a4f-9f9f-2b3a2369ec58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:34.387090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-2ae2a184-5abb-4a4f-9f9f-2b3a2369ec58 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:34.389862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2ae2a184-5abb-4a4f-9f9f-2b3a2369ec58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:34.442147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78bf2e-ba6e-45fa-84d3-1a30c975a15b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:34.466841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 913/1845] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:34.476325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c1427-eb38-4dd2-9be1-2a97e8da3ca1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:34.476616 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c1427-eb38-4dd2-9be1-2a97e8da3ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:34.476901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c1427-eb38-4dd2-9be1-2a97e8da3ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:34.483532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:34.518071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-cc1c1427-eb38-4dd2-9be1-2a97e8da3ca1 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:34.519006 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cc1c1427-eb38-4dd2-9be1-2a97e8da3ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:34.525266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:34.525940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:34.526225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:34.526727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] PATH_INFO: `/v3/users/d46931fa823b40a4b1171ea57a57b121` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:34.528697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d46931fa823b40a4b1171ea57a57b121)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:34.531577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:34.533211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 914/1846] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:34.536717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c1427-eb38-4dd2-9be1-2a97e8da3ca1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:34.565034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c1427-eb38-4dd2-9be1-2a97e8da3ca1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:34.580172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b2685-96e9-4830-a136-39a691d89d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:34.580847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b2685-96e9-4830-a136-39a691d89d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:34.581282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b2685-96e9-4830-a136-39a691d89d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:34.581705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b2685-96e9-4830-a136-39a691d89d24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:34.585006 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b2685-96e9-4830-a136-39a691d89d24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:30:34.585437 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b2685-96e9-4830-a136-39a691d89d24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:34.588634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b2685-96e9-4830-a136-39a691d89d24 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:34.621655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 915/1847] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:30:34.627632 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b2685-96e9-4830-a136-39a691d89d24 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:34.634146 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd46931fa823b40a4b1171ea57a57b121'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.635444 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd46931fa823b40a4b1171ea57a57b121'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.645940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd46931fa823b40a4b1171ea57a57b121'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.647591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b2685-96e9-4830-a136-39a691d89d24 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:34.658257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b44d15-0277-47e2-a7f5-3632165cda65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:34.658742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd46931fa823b40a4b1171ea57a57b121'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.663547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b44d15-0277-47e2-a7f5-3632165cda65 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:34.663547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b44d15-0277-47e2-a7f5-3632165cda65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:34.667567 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b44d15-0277-47e2-a7f5-3632165cda65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:34.668389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b44d15-0277-47e2-a7f5-3632165cda65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:34.674099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b44d15-0277-47e2-a7f5-3632165cda65 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:34.683381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd46931fa823b40a4b1171ea57a57b121'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:34.695274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 933/1848] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:34 2022] DELETE /identity/v3/users/d46931fa823b40a4b1171ea57a57b121 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:34.739191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 916/1849] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:34 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:30:34.741175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:34.742886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:34.843708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b0bc3-106b-4dff-8c58-b2fde3502e30 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:34.964681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf9cf1-5380-409c-a0bf-9ecde2e7903a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:34.968796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf9cf1-5380-409c-a0bf-9ecde2e7903a admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:34.971536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf9cf1-5380-409c-a0bf-9ecde2e7903a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:34.972168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf9cf1-5380-409c-a0bf-9ecde2e7903a admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:34.974018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf9cf1-5380-409c-a0bf-9ecde2e7903a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:34.977813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf9cf1-5380-409c-a0bf-9ecde2e7903a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:35.005697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:35.043990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 934/1850] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:34 2022] POST /identity/v3/users => generated 289 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:35.048879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b44d15-0277-47e2-a7f5-3632165cda65 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:35.067304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b44d15-0277-47e2-a7f5-3632165cda65 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:35.078311 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916383fb-8933-4a15-9294-efdff1dd816b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:35.079411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916383fb-8933-4a15-9294-efdff1dd816b admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:35.079909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916383fb-8933-4a15-9294-efdff1dd816b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:35.080443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916383fb-8933-4a15-9294-efdff1dd816b admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:35.081312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916383fb-8933-4a15-9294-efdff1dd816b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:35.083281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916383fb-8933-4a15-9294-efdff1dd816b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:35.120374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 917/1851] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:35.123521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf9cf1-5380-409c-a0bf-9ecde2e7903a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:35.144471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf9cf1-5380-409c-a0bf-9ecde2e7903a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:35.154118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb87eb-e18f-4240-8831-1ef014eb702c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:35.155416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb87eb-e18f-4240-8831-1ef014eb702c admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:35.155913 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb87eb-e18f-4240-8831-1ef014eb702c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:35.156721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb87eb-e18f-4240-8831-1ef014eb702c admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:35.157648 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb87eb-e18f-4240-8831-1ef014eb702c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:35.159534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb87eb-e18f-4240-8831-1ef014eb702c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:35.177809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 935/1852] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:35.181920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916383fb-8933-4a15-9294-efdff1dd816b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:35.195705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916383fb-8933-4a15-9294-efdff1dd816b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:35.218806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fd1e2-2f16-4225-830f-e56db11ce94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:35.218806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fd1e2-2f16-4225-830f-e56db11ce94c admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:35.218806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fd1e2-2f16-4225-830f-e56db11ce94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:35.218806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fd1e2-2f16-4225-830f-e56db11ce94c admin admin] PATH_INFO: `/v3/projects/8c1ecca24a3f47579024d2e60c6c21d4/users/06a9bffefc614671a70315e6f994502f/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:35.261291 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fd1e2-2f16-4225-830f-e56db11ce94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c1ecca24a3f47579024d2e60c6c21d4, user_id=06a9bffefc614671a70315e6f994502f, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:35.265815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fd1e2-2f16-4225-830f-e56db11ce94c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:35.307917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fa37c-08ca-45ba-b93c-8fe63559a573 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:35.308372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 918/1853] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:35 2022] PUT /identity/v3/projects/8c1ecca24a3f47579024d2e60c6c21d4/users/06a9bffefc614671a70315e6f994502f/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:35.310255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fa37c-08ca-45ba-b93c-8fe63559a573 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:35.312844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fa37c-08ca-45ba-b93c-8fe63559a573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:35.388556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2f6fa37c-08ca-45ba-b93c-8fe63559a573 None None] MFA Rules not processed for user `06a9bffefc614671a70315e6f994502f`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:35.394869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2f6fa37c-08ca-45ba-b93c-8fe63559a573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:35.513558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 936/1854] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:35.526238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fd1e2-2f16-4225-830f-e56db11ce94c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:35.528906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2b2fd1e2-2f16-4225-830f-e56db11ce94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:35.612702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fd1e2-2f16-4225-830f-e56db11ce94c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:35.618085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43122908-fbd7-4f52-8b3b-c667cb57f6c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0330c79c391d463b91f99e030c6dcc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:35.618544 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43122908-fbd7-4f52-8b3b-c667cb57f6c3 service glance] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:35.618897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43122908-fbd7-4f52-8b3b-c667cb57f6c3 service glance] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:35.621082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43122908-fbd7-4f52-8b3b-c667cb57f6c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:35.634875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43122908-fbd7-4f52-8b3b-c667cb57f6c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:35.639354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43122908-fbd7-4f52-8b3b-c667cb57f6c3 service glance] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:35.667567 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 919/1855] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:35.699132 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fa37c-08ca-45ba-b93c-8fe63559a573 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:35.703817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2f6fa37c-08ca-45ba-b93c-8fe63559a573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:35.856585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fa37c-08ca-45ba-b93c-8fe63559a573 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:35.868180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a52d80-b333-4ff7-82ab-4d04eb31d62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:35.868832 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a52d80-b333-4ff7-82ab-4d04eb31d62f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:35.869115 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a52d80-b333-4ff7-82ab-4d04eb31d62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:35.869387 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a52d80-b333-4ff7-82ab-4d04eb31d62f admin admin] PATH_INFO: `/v3/projects/de09720fe1b44d60ba0043da6668239a` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:35.873675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a52d80-b333-4ff7-82ab-4d04eb31d62f admin admin] RBAC: Authorizing `identity:delete_project(project_id=de09720fe1b44d60ba0043da6668239a)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:35.875563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a52d80-b333-4ff7-82ab-4d04eb31d62f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:35.944820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-34a52d80-b333-4ff7-82ab-4d04eb31d62f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de09720fe1b44d60ba0043da6668239a', 'request_id': 'req-34a52d80-b333-4ff7-82ab-4d04eb31d62f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:35.950772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-34a52d80-b333-4ff7-82ab-4d04eb31d62f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de09720fe1b44d60ba0043da6668239a', 'request_id': 'req-34a52d80-b333-4ff7-82ab-4d04eb31d62f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:35.959092 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-34a52d80-b333-4ff7-82ab-4d04eb31d62f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de09720fe1b44d60ba0043da6668239a', 'request_id': 'req-34a52d80-b333-4ff7-82ab-4d04eb31d62f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:35.974096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:35.982771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-34a52d80-b333-4ff7-82ab-4d04eb31d62f admin admin] The token cache is being invalidate because project de09720fe1b44d60ba0043da6668239a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:35.983135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-34a52d80-b333-4ff7-82ab-4d04eb31d62f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:35.989028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-34a52d80-b333-4ff7-82ab-4d04eb31d62f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:35.990075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 937/1856] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:35 2022] DELETE /identity/v3/projects/de09720fe1b44d60ba0043da6668239a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:35.995090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb184d4-b698-475c-b561-1c3ea0c74282 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:35.995374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb184d4-b698-475c-b561-1c3ea0c74282 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:35.995648 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb184d4-b698-475c-b561-1c3ea0c74282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:36.005637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:36.038588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-cfb184d4-b698-475c-b561-1c3ea0c74282 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:36.041076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cfb184d4-b698-475c-b561-1c3ea0c74282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:36.221107 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 920/1857] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:36.230024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7d878-0600-4e38-91a9-2852569ef9f1 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:36.230024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7d878-0600-4e38-91a9-2852569ef9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:36.230024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7d878-0600-4e38-91a9-2852569ef9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:36.291062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-fbb7d878-0600-4e38-91a9-2852569ef9f1 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:36.292737 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fbb7d878-0600-4e38-91a9-2852569ef9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:36.327646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 938/1858] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:36.328927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb184d4-b698-475c-b561-1c3ea0c74282 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:36.343683 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb184d4-b698-475c-b561-1c3ea0c74282 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:36.365013 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d905f-c364-4209-bee2-9b8433417ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:36.368476 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d905f-c364-4209-bee2-9b8433417ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:36.368839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d905f-c364-4209-bee2-9b8433417ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:36.369428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d905f-c364-4209-bee2-9b8433417ed7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:36.370306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d905f-c364-4209-bee2-9b8433417ed7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:30:36.370754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d905f-c364-4209-bee2-9b8433417ed7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:36.377756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d905f-c364-4209-bee2-9b8433417ed7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:36.418693 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 921/1859] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:30:36.423837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7d878-0600-4e38-91a9-2852569ef9f1 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:36.434977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7d878-0600-4e38-91a9-2852569ef9f1 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:36.447719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ec902-ec65-4162-bd2b-6024ab9977ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:36.448692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec902-ec65-4162-bd2b-6024ab9977ad admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:36.449407 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec902-ec65-4162-bd2b-6024ab9977ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:36.449931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec902-ec65-4162-bd2b-6024ab9977ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:36.457694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ec902-ec65-4162-bd2b-6024ab9977ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:36.460305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ec902-ec65-4162-bd2b-6024ab9977ad admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:36.541125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 939/1860] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:36 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:36.547305 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d905f-c364-4209-bee2-9b8433417ed7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:36.566903 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d905f-c364-4209-bee2-9b8433417ed7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:36.692792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a19f6-ed6b-4dca-9c5a-dc0df0ed5d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:36.693421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a19f6-ed6b-4dca-9c5a-dc0df0ed5d89 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:36.693708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a19f6-ed6b-4dca-9c5a-dc0df0ed5d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:36.693985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a19f6-ed6b-4dca-9c5a-dc0df0ed5d89 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:36.695584 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ec902-ec65-4162-bd2b-6024ab9977ad admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:36.697940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a19f6-ed6b-4dca-9c5a-dc0df0ed5d89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:36.698069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-be6ec902-ec65-4162-bd2b-6024ab9977ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:36.703467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a19f6-ed6b-4dca-9c5a-dc0df0ed5d89 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:36.720754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ec902-ec65-4162-bd2b-6024ab9977ad admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:36.734806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9eb69-68e8-4e45-b8ee-848ad9cd3aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:36.735817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9eb69-68e8-4e45-b8ee-848ad9cd3aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:36.736152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9eb69-68e8-4e45-b8ee-848ad9cd3aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:36.736428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9eb69-68e8-4e45-b8ee-848ad9cd3aed admin admin] PATH_INFO: `/v3/users/9022d7f303324ccbb2cbd84e30b7fe47` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:36.737856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9eb69-68e8-4e45-b8ee-848ad9cd3aed admin admin] RBAC: Authorizing `identity:delete_user(user_id=9022d7f303324ccbb2cbd84e30b7fe47)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:36.742358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9eb69-68e8-4e45-b8ee-848ad9cd3aed admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:36.752528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 922/1861] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:36 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:36.756821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a19f6-ed6b-4dca-9c5a-dc0df0ed5d89 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:36.770860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a19f6-ed6b-4dca-9c5a-dc0df0ed5d89 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:36.785869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ae518-8066-477e-a792-77ec477993bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:36.787427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ae518-8066-477e-a792-77ec477993bd admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:36.787666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ae518-8066-477e-a792-77ec477993bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:36.787953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ae518-8066-477e-a792-77ec477993bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:36.788804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ae518-8066-477e-a792-77ec477993bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:36.790677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ae518-8066-477e-a792-77ec477993bd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:36.808520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 923/1862] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:36.811752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ae518-8066-477e-a792-77ec477993bd admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:36.827317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9ef9eb69-68e8-4e45-b8ee-848ad9cd3aed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9022d7f303324ccbb2cbd84e30b7fe47'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:36.828315 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9ef9eb69-68e8-4e45-b8ee-848ad9cd3aed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9022d7f303324ccbb2cbd84e30b7fe47'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:36.836415 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ae518-8066-477e-a792-77ec477993bd admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:36.838653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9ef9eb69-68e8-4e45-b8ee-848ad9cd3aed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9022d7f303324ccbb2cbd84e30b7fe47'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:36.852468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ee2b0-1615-43df-aa8d-635b52d174e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:36.853528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9ef9eb69-68e8-4e45-b8ee-848ad9cd3aed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9022d7f303324ccbb2cbd84e30b7fe47'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:36.854088 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ee2b0-1615-43df-aa8d-635b52d174e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:36.855432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ee2b0-1615-43df-aa8d-635b52d174e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:36.856055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ee2b0-1615-43df-aa8d-635b52d174e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:36.857895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ee2b0-1615-43df-aa8d-635b52d174e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:36.865937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ee2b0-1615-43df-aa8d-635b52d174e9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:36.886428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9ef9eb69-68e8-4e45-b8ee-848ad9cd3aed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9022d7f303324ccbb2cbd84e30b7fe47'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:36.890352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 924/1863] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:36.895247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ee2b0-1615-43df-aa8d-635b52d174e9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:36.896970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-da7ee2b0-1615-43df-aa8d-635b52d174e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:36.912392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 940/1864] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:36 2022] DELETE /identity/v3/users/9022d7f303324ccbb2cbd84e30b7fe47 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:36.916325 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a65b91-d462-4e86-b138-dfe6db8af391 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:36.916762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a65b91-d462-4e86-b138-dfe6db8af391 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:36.917052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a65b91-d462-4e86-b138-dfe6db8af391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:36.919933 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ee2b0-1615-43df-aa8d-635b52d174e9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:36.950866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-87a65b91-d462-4e86-b138-dfe6db8af391 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:36.952215 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-87a65b91-d462-4e86-b138-dfe6db8af391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:37.066341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:37.084241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e595d68-3590-46ea-8281-cc46379031fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:37.084683 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e595d68-3590-46ea-8281-cc46379031fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:37.084961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e595d68-3590-46ea-8281-cc46379031fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:37.085235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e595d68-3590-46ea-8281-cc46379031fa admin admin] PATH_INFO: `/v3/projects/d4562935f6bd403c93d0ed68f64dc98c/users/7d22c0ca830c47c7847c88ee1b74129b/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:37.095520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 941/1865] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:37.110317 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e595d68-3590-46ea-8281-cc46379031fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4562935f6bd403c93d0ed68f64dc98c, user_id=7d22c0ca830c47c7847c88ee1b74129b, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:37.113563 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e595d68-3590-46ea-8281-cc46379031fa admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:37.145388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 925/1866] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:36 2022] PUT /identity/v3/projects/d4562935f6bd403c93d0ed68f64dc98c/users/7d22c0ca830c47c7847c88ee1b74129b/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:37.173297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e595d68-3590-46ea-8281-cc46379031fa admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:37.177194 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3e595d68-3590-46ea-8281-cc46379031fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:37.177862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a76c2-352d-45c3-b27d-d2b4b414d10a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:37.178370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a76c2-352d-45c3-b27d-d2b4b414d10a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:37.178688 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a76c2-352d-45c3-b27d-d2b4b414d10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:37.250006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bf8a76c2-352d-45c3-b27d-d2b4b414d10a None None] MFA Rules not processed for user `7d22c0ca830c47c7847c88ee1b74129b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:37.252126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bf8a76c2-352d-45c3-b27d-d2b4b414d10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:37.337314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:37.340221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e595d68-3590-46ea-8281-cc46379031fa admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:37.345889 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fee43-8ebe-4567-975b-75c2f66f5d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:37.349539 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fee43-8ebe-4567-975b-75c2f66f5d9d service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:37.350683 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fee43-8ebe-4567-975b-75c2f66f5d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:37.351920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fee43-8ebe-4567-975b-75c2f66f5d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:37.429280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 942/1867] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:37.435703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a76c2-352d-45c3-b27d-d2b4b414d10a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:37.528646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fee43-8ebe-4567-975b-75c2f66f5d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:37.529852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fee43-8ebe-4567-975b-75c2f66f5d9d service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:37.563631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 926/1868] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:37.674691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a76c2-352d-45c3-b27d-d2b4b414d10a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:37.705483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5262dc-0bd8-4685-86e2-0d69b84ea1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:37.706128 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5262dc-0bd8-4685-86e2-0d69b84ea1cb admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:37.706411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5262dc-0bd8-4685-86e2-0d69b84ea1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:37.706693 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5262dc-0bd8-4685-86e2-0d69b84ea1cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:37.709891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5262dc-0bd8-4685-86e2-0d69b84ea1cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:37.713141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5262dc-0bd8-4685-86e2-0d69b84ea1cb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:37.784922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fee43-8ebe-4567-975b-75c2f66f5d9d service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:37.785346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 943/1869] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:37 2022] POST /identity/v3/projects => generated 383 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:37.815979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fee43-8ebe-4567-975b-75c2f66f5d9d service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:37.968714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b2b78-30ac-4e63-8832-d9ef0fbbb2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:37.969448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b2b78-30ac-4e63-8832-d9ef0fbbb2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:37.969793 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b2b78-30ac-4e63-8832-d9ef0fbbb2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:37.970130 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b2b78-30ac-4e63-8832-d9ef0fbbb2e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:37.970852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b2b78-30ac-4e63-8832-d9ef0fbbb2e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:37.972901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b2b78-30ac-4e63-8832-d9ef0fbbb2e5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:37.996171 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda3c1d-cd6c-4481-9f44-4448a78c1d89 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:37.996790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda3c1d-cd6c-4481-9f44-4448a78c1d89 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:37.997136 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda3c1d-cd6c-4481-9f44-4448a78c1d89 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:38.001263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 944/1870] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 05:30:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:30:38.018002 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 927/1871] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:37 2022] POST /identity/v3/users => generated 298 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:38.022037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda3c1d-cd6c-4481-9f44-4448a78c1d89 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:38.022583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063cf4e-d7a1-400f-8953-9b5462e36f92 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:38.022876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063cf4e-d7a1-400f-8953-9b5462e36f92 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:38.023192 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063cf4e-d7a1-400f-8953-9b5462e36f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:38.070984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda3c1d-cd6c-4481-9f44-4448a78c1d89 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:38.087018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77aadd8-3deb-45c5-a7eb-4d9f86440222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:38.087641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77aadd8-3deb-45c5-a7eb-4d9f86440222 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:38.087926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77aadd8-3deb-45c5-a7eb-4d9f86440222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:38.088600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77aadd8-3deb-45c5-a7eb-4d9f86440222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:38.090178 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77aadd8-3deb-45c5-a7eb-4d9f86440222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:38.092364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77aadd8-3deb-45c5-a7eb-4d9f86440222 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:38.100217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7063cf4e-d7a1-400f-8953-9b5462e36f92 None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:38.109919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7063cf4e-d7a1-400f-8953-9b5462e36f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:38.138701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 945/1872] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:38.140074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77aadd8-3deb-45c5-a7eb-4d9f86440222 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:38.157224 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:38.189516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77aadd8-3deb-45c5-a7eb-4d9f86440222 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:38.206757 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752d08d-5925-45d0-891a-3bafedbf7d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:38.208970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752d08d-5925-45d0-891a-3bafedbf7d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:38.209454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752d08d-5925-45d0-891a-3bafedbf7d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:38.210531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752d08d-5925-45d0-891a-3bafedbf7d94 admin admin] PATH_INFO: `/v3/projects/517a508ed697414ea0f66abc8f149988/users/07e7033b2ca64a9892549cb611ebd3a2/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:38.312180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 928/1873] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 05:30:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:38.323224 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752d08d-5925-45d0-891a-3bafedbf7d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=517a508ed697414ea0f66abc8f149988, user_id=07e7033b2ca64a9892549cb611ebd3a2, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:38.325978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752d08d-5925-45d0-891a-3bafedbf7d94 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:38.352042 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 946/1874] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:38 2022] PUT /identity/v3/projects/517a508ed697414ea0f66abc8f149988/users/07e7033b2ca64a9892549cb611ebd3a2/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:38.356185 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063cf4e-d7a1-400f-8953-9b5462e36f92 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:38.383247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063cf4e-d7a1-400f-8953-9b5462e36f92 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:38.479937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752d08d-5925-45d0-891a-3bafedbf7d94 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:38.482908 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f752d08d-5925-45d0-891a-3bafedbf7d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:38.548572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee5d59-9e8f-4869-b09e-60ee8423e3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:38.549529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee5d59-9e8f-4869-b09e-60ee8423e3ff admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:38.549956 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee5d59-9e8f-4869-b09e-60ee8423e3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:38.550329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee5d59-9e8f-4869-b09e-60ee8423e3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:38.551136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee5d59-9e8f-4869-b09e-60ee8423e3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:38.553303 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee5d59-9e8f-4869-b09e-60ee8423e3ff admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:38.576411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 929/1875] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:38.577666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee5d59-9e8f-4869-b09e-60ee8423e3ff admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:38.581911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-93ee5d59-9e8f-4869-b09e-60ee8423e3ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:38.617705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee5d59-9e8f-4869-b09e-60ee8423e3ff admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:38.631687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:38.632516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:38.633282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:38.633724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] PATH_INFO: `/v3/projects/de31030d2e5b4c2abe5b01262bc0b86a` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:38.634922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de31030d2e5b4c2abe5b01262bc0b86a)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:38.646029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:38.666639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752d08d-5925-45d0-891a-3bafedbf7d94 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:38.674129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461330b1-bb01-4e7f-9b1b-92f1a2980453 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:38.674941 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461330b1-bb01-4e7f-9b1b-92f1a2980453 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:38.675674 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461330b1-bb01-4e7f-9b1b-92f1a2980453 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:38.676051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461330b1-bb01-4e7f-9b1b-92f1a2980453 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:38.688461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461330b1-bb01-4e7f-9b1b-92f1a2980453 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:38.690670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461330b1-bb01-4e7f-9b1b-92f1a2980453 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:38.729318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 947/1876] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:38.730291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461330b1-bb01-4e7f-9b1b-92f1a2980453 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:38.750882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461330b1-bb01-4e7f-9b1b-92f1a2980453 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:38.756815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de31030d2e5b4c2abe5b01262bc0b86a', 'request_id': 'req-b1185396-d9b8-41d2-9fd9-742baa640797'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:38.760531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:38.787011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de31030d2e5b4c2abe5b01262bc0b86a', 'request_id': 'req-b1185396-d9b8-41d2-9fd9-742baa640797'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:38.800844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de31030d2e5b4c2abe5b01262bc0b86a', 'request_id': 'req-b1185396-d9b8-41d2-9fd9-742baa640797'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:38.819948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] The token cache is being invalidate because project de31030d2e5b4c2abe5b01262bc0b86a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:38.820470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:38.821295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:38.826534 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 930/1877] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:38 2022] DELETE /identity/v3/projects/de31030d2e5b4c2abe5b01262bc0b86a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:38.827587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:38.829029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:38.847199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99b787-f89b-4789-a3e1-165e095c1b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:38.847874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99b787-f89b-4789-a3e1-165e095c1b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:38.848181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99b787-f89b-4789-a3e1-165e095c1b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:38.848518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99b787-f89b-4789-a3e1-165e095c1b71 admin admin] PATH_INFO: `/v3/projects/517a508ed697414ea0f66abc8f149988/users/07e7033b2ca64a9892549cb611ebd3a2/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:38.850382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99b787-f89b-4789-a3e1-165e095c1b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=517a508ed697414ea0f66abc8f149988, user_id=07e7033b2ca64a9892549cb611ebd3a2, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:38.853246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99b787-f89b-4789-a3e1-165e095c1b71 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:38.868550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 948/1878] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:38 2022] PUT /identity/v3/projects/517a508ed697414ea0f66abc8f149988/users/07e7033b2ca64a9892549cb611ebd3a2/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:38.871874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:38.880341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bf962-e2ea-45bf-9a47-75b7de8b8249 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:38.880650 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bf962-e2ea-45bf-9a47-75b7de8b8249 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:38.880896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bf962-e2ea-45bf-9a47-75b7de8b8249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:38.911575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1185396-d9b8-41d2-9fd9-742baa640797 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:38.928966 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b07bf962-e2ea-45bf-9a47-75b7de8b8249 None None] MFA Rules not processed for user `07e7033b2ca64a9892549cb611ebd3a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:38.930989 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b07bf962-e2ea-45bf-9a47-75b7de8b8249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:39.015378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:39.017058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:39.017593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:39.018056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] PATH_INFO: `/v3/users/bc9a92bae63e49dc950acf13d351570a` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:39.021704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc9a92bae63e49dc950acf13d351570a)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:39.022394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 949/1879] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:39.025038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:39.040506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bf962-e2ea-45bf-9a47-75b7de8b8249 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:39.047109 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b07bf962-e2ea-45bf-9a47-75b7de8b8249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:39.129901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc9a92bae63e49dc950acf13d351570a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:39.134860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bf962-e2ea-45bf-9a47-75b7de8b8249 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:39.142600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5add8-c2b8-47a9-9d33-425c88ebad07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:39.143330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5add8-c2b8-47a9-9d33-425c88ebad07 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:39.143622 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5add8-c2b8-47a9-9d33-425c88ebad07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:39.143987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5add8-c2b8-47a9-9d33-425c88ebad07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:39.151113 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc9a92bae63e49dc950acf13d351570a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:39.156173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5add8-c2b8-47a9-9d33-425c88ebad07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:39.157561 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5add8-c2b8-47a9-9d33-425c88ebad07 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:39.168092 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 950/1880] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:30:39.195225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc9a92bae63e49dc950acf13d351570a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:39.197163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc9a92bae63e49dc950acf13d351570a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:39.198382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc9a92bae63e49dc950acf13d351570a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:39.224513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 931/1881] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:38 2022] DELETE /identity/v3/users/bc9a92bae63e49dc950acf13d351570a => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:39.327251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5add8-c2b8-47a9-9d33-425c88ebad07 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:39.339539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5add8-c2b8-47a9-9d33-425c88ebad07 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:39.429527 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:39.488300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b7b61-eb45-455f-a013-80c4dd661404 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:39.488923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b7b61-eb45-455f-a013-80c4dd661404 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:39.489204 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b7b61-eb45-455f-a013-80c4dd661404 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:39.489473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b7b61-eb45-455f-a013-80c4dd661404 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:39.491847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1b1b7b61-eb45-455f-a013-80c4dd661404 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:39.627710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:39.637396 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:39.647421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b7b61-eb45-455f-a013-80c4dd661404 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:39.657804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b7b61-eb45-455f-a013-80c4dd661404 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:39.695866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 951/1882] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:39.824662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:39.839246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea4a5c-648a-409f-9640-e2ac45f6e58e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:39.846870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:39.847633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:39.848059 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:39.848428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] PATH_INFO: `/v3/projects/ecc183484b0d482d96f8697895a78eb9` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:39.849863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecc183484b0d482d96f8697895a78eb9)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:39.852031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:39.964221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecc183484b0d482d96f8697895a78eb9', 'request_id': 'req-be835b6f-2681-48c4-bb84-b57ed0eaa437'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:39.983813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecc183484b0d482d96f8697895a78eb9', 'request_id': 'req-be835b6f-2681-48c4-bb84-b57ed0eaa437'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.004656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecc183484b0d482d96f8697895a78eb9', 'request_id': 'req-be835b6f-2681-48c4-bb84-b57ed0eaa437'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.039129 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] The token cache is being invalidate because project ecc183484b0d482d96f8697895a78eb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:40.039129 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.040399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.043193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 932/1883] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:39 2022] DELETE /identity/v3/projects/ecc183484b0d482d96f8697895a78eb9 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:40.048248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b7b61-eb45-455f-a013-80c4dd661404 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:40.067120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b7b61-eb45-455f-a013-80c4dd661404 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:40.174974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:40.176832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:40.179482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:40.180327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:40.180604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:40.180880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] PATH_INFO: `/v3/users/64a2534b615b41f58f57e4e0bd598c5d` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:40.182203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a2534b615b41f58f57e4e0bd598c5d)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:40.184557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:40.200947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be835b6f-2681-48c4-bb84-b57ed0eaa437 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:40.213798 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5024ba-fa74-4ce1-9ac0-b9a727657fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:40.214462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5024ba-fa74-4ce1-9ac0-b9a727657fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:40.214738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5024ba-fa74-4ce1-9ac0-b9a727657fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:40.215010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5024ba-fa74-4ce1-9ac0-b9a727657fcd admin admin] PATH_INFO: `/v3/users/7d22c0ca830c47c7847c88ee1b74129b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:40.224101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5024ba-fa74-4ce1-9ac0-b9a727657fcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d22c0ca830c47c7847c88ee1b74129b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:40.226061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5024ba-fa74-4ce1-9ac0-b9a727657fcd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:40.282654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a2534b615b41f58f57e4e0bd598c5d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.283249 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a2534b615b41f58f57e4e0bd598c5d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.298837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0e5024ba-fa74-4ce1-9ac0-b9a727657fcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d22c0ca830c47c7847c88ee1b74129b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.305171 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a2534b615b41f58f57e4e0bd598c5d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.308032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0e5024ba-fa74-4ce1-9ac0-b9a727657fcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d22c0ca830c47c7847c88ee1b74129b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.323973 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a2534b615b41f58f57e4e0bd598c5d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.328267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0e5024ba-fa74-4ce1-9ac0-b9a727657fcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d22c0ca830c47c7847c88ee1b74129b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.329248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0e5024ba-fa74-4ce1-9ac0-b9a727657fcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d22c0ca830c47c7847c88ee1b74129b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.329873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0e5024ba-fa74-4ce1-9ac0-b9a727657fcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d22c0ca830c47c7847c88ee1b74129b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.350088 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 933/1884] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:40 2022] DELETE /identity/v3/users/7d22c0ca830c47c7847c88ee1b74129b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:40.355685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977c2d9-c8a8-4f9d-aa63-650e71bffdea None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:40.355970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977c2d9-c8a8-4f9d-aa63-650e71bffdea None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:40.356105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a2534b615b41f58f57e4e0bd598c5d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:40.356532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977c2d9-c8a8-4f9d-aa63-650e71bffdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:40.372312 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 952/1885] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:40 2022] DELETE /identity/v3/users/64a2534b615b41f58f57e4e0bd598c5d => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:40.392662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8977c2d9-c8a8-4f9d-aa63-650e71bffdea None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:40.409906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8977c2d9-c8a8-4f9d-aa63-650e71bffdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:40.486753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:40.501708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 934/1886] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:40.530745 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:40.532548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:40.541862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:40.606229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edbfaaa-320b-48d9-aacb-8c3f798ad7cd admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:40.613279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166a37a-7ee4-4672-8423-ad539447358b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:40.617543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166a37a-7ee4-4672-8423-ad539447358b service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:40.618263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166a37a-7ee4-4672-8423-ad539447358b service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:40.618758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166a37a-7ee4-4672-8423-ad539447358b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:40.634902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166a37a-7ee4-4672-8423-ad539447358b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:40.636634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166a37a-7ee4-4672-8423-ad539447358b service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:40.661933 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 953/1887] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:41.372163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977c2d9-c8a8-4f9d-aa63-650e71bffdea None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:41.374251 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8977c2d9-c8a8-4f9d-aa63-650e71bffdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:41.391610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977c2d9-c8a8-4f9d-aa63-650e71bffdea None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:41.397716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:41.398669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:41.398810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:41.399157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e admin admin] PATH_INFO: `/v3/projects/40f4eb6387014bcc9b3270cda947aeff` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:41.400525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e admin admin] RBAC: Authorizing `identity:delete_project(project_id=40f4eb6387014bcc9b3270cda947aeff)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:41.402544 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:41.461265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166a37a-7ee4-4672-8423-ad539447358b service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:41.463503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8166a37a-7ee4-4672-8423-ad539447358b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:41.472732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40f4eb6387014bcc9b3270cda947aeff', 'request_id': 'req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.481629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166a37a-7ee4-4672-8423-ad539447358b service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:41.481738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40f4eb6387014bcc9b3270cda947aeff', 'request_id': 'req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.490301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40f4eb6387014bcc9b3270cda947aeff', 'request_id': 'req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.503967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e admin admin] The token cache is being invalidate because project 40f4eb6387014bcc9b3270cda947aeff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:41.504289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.504954 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1f6245b9-f3d0-4f2e-aa56-2247b865e78e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.506465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 935/1888] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:41 2022] DELETE /identity/v3/projects/40f4eb6387014bcc9b3270cda947aeff => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:41.511423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be068eb6-2a8a-4ecf-a886-f88778ae1fd7 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:41.511734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be068eb6-2a8a-4ecf-a886-f88778ae1fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:41.511986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be068eb6-2a8a-4ecf-a886-f88778ae1fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:41.542687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-be068eb6-2a8a-4ecf-a886-f88778ae1fd7 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:41.544213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-be068eb6-2a8a-4ecf-a886-f88778ae1fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:41.551697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:41.552423 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:41.552952 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:41.553234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] PATH_INFO: `/v3/projects/d4562935f6bd403c93d0ed68f64dc98c` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:41.554924 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4562935f6bd403c93d0ed68f64dc98c)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:41.556916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:41.606949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 936/1889] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:41.611931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4562935f6bd403c93d0ed68f64dc98c', 'request_id': 'req-dd0f3269-72e1-4398-875d-0b4fb63beb62'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.615769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee81d2-b18e-4f60-9531-cee90393ae23 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:41.616089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee81d2-b18e-4f60-9531-cee90393ae23 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:41.616365 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee81d2-b18e-4f60-9531-cee90393ae23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:41.619337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4562935f6bd403c93d0ed68f64dc98c', 'request_id': 'req-dd0f3269-72e1-4398-875d-0b4fb63beb62'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.627625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4562935f6bd403c93d0ed68f64dc98c', 'request_id': 'req-dd0f3269-72e1-4398-875d-0b4fb63beb62'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.644574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-bbee81d2-b18e-4f60-9531-cee90393ae23 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:41.644574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bbee81d2-b18e-4f60-9531-cee90393ae23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:41.645911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] The token cache is being invalidate because project d4562935f6bd403c93d0ed68f64dc98c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:41.646317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.647195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.648829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 954/1890] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:41 2022] DELETE /identity/v3/projects/d4562935f6bd403c93d0ed68f64dc98c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:41.652577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:41.654009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:41.698550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:41.713115 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 937/1891] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:41.717200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee81d2-b18e-4f60-9531-cee90393ae23 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:41.726644 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee81d2-b18e-4f60-9531-cee90393ae23 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:41.727747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f3269-72e1-4398-875d-0b4fb63beb62 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:41.731517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a88ea-8abb-4e63-8925-b912a98cfae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:41.732177 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a88ea-8abb-4e63-8925-b912a98cfae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:41.733124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a88ea-8abb-4e63-8925-b912a98cfae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:41.733509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a88ea-8abb-4e63-8925-b912a98cfae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:41.733724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:41.734165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a88ea-8abb-4e63-8925-b912a98cfae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:30:41.734551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a88ea-8abb-4e63-8925-b912a98cfae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:41.734732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:41.735080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:41.735406 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] PATH_INFO: `/v3/users/07e7033b2ca64a9892549cb611ebd3a2` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:41.736639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a88ea-8abb-4e63-8925-b912a98cfae1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:41.736765 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e7033b2ca64a9892549cb611ebd3a2)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:41.738826 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:41.756413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 938/1892] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:30:41.761111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a88ea-8abb-4e63-8925-b912a98cfae1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:41.777959 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a88ea-8abb-4e63-8925-b912a98cfae1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:41.785349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c60ef0-8df2-4bb4-acaa-986fba2eecb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:41.786228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c60ef0-8df2-4bb4-acaa-986fba2eecb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:41.786704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c60ef0-8df2-4bb4-acaa-986fba2eecb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:41.787082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c60ef0-8df2-4bb4-acaa-986fba2eecb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:41.787899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c60ef0-8df2-4bb4-acaa-986fba2eecb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:41.791527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c60ef0-8df2-4bb4-acaa-986fba2eecb2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:41.802023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:41.827105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 939/1893] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:41 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:30:41.831434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c60ef0-8df2-4bb4-acaa-986fba2eecb2 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:41.834974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e7033b2ca64a9892549cb611ebd3a2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.835896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e7033b2ca64a9892549cb611ebd3a2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.846763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e7033b2ca64a9892549cb611ebd3a2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.871816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e7033b2ca64a9892549cb611ebd3a2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.905937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e7033b2ca64a9892549cb611ebd3a2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:41.915293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 955/1894] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:41 2022] DELETE /identity/v3/users/07e7033b2ca64a9892549cb611ebd3a2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:41.977888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c60ef0-8df2-4bb4-acaa-986fba2eecb2 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:42.120799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef9e60-6060-4e56-8a84-161eef1aa7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:42.121910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef9e60-6060-4e56-8a84-161eef1aa7bb admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.121910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef9e60-6060-4e56-8a84-161eef1aa7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.122141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef9e60-6060-4e56-8a84-161eef1aa7bb admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.122594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef9e60-6060-4e56-8a84-161eef1aa7bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:42.125204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef9e60-6060-4e56-8a84-161eef1aa7bb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:42.149449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 940/1895] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:41 2022] POST /identity/v3/users => generated 291 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:42.153299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:42.155255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:42.193029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d88ef6-17a1-4733-adcb-600e720d35de admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:42.199611 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ad1f0-f374-4d97-b3e1-4d08cd5a4844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:42.200378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ad1f0-f374-4d97-b3e1-4d08cd5a4844 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.200726 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ad1f0-f374-4d97-b3e1-4d08cd5a4844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.208280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ad1f0-f374-4d97-b3e1-4d08cd5a4844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.209131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ad1f0-f374-4d97-b3e1-4d08cd5a4844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:42.212122 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ad1f0-f374-4d97-b3e1-4d08cd5a4844 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:42.223380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 956/1896] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:42.228034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef9e60-6060-4e56-8a84-161eef1aa7bb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:42.238859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef9e60-6060-4e56-8a84-161eef1aa7bb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:42.243711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c9e71-7c4b-43f6-8d40-e8f90e2caebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:42.244500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c9e71-7c4b-43f6-8d40-e8f90e2caebc admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.244883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c9e71-7c4b-43f6-8d40-e8f90e2caebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.245734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c9e71-7c4b-43f6-8d40-e8f90e2caebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.245989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c9e71-7c4b-43f6-8d40-e8f90e2caebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:42.247702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c9e71-7c4b-43f6-8d40-e8f90e2caebc admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:42.260106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 941/1897] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:42.264416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ad1f0-f374-4d97-b3e1-4d08cd5a4844 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:42.274618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ad1f0-f374-4d97-b3e1-4d08cd5a4844 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:42.280184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69b7c7-bf15-49a4-9ffe-1bcc2bf9d3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:42.280979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69b7c7-bf15-49a4-9ffe-1bcc2bf9d3fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.281341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69b7c7-bf15-49a4-9ffe-1bcc2bf9d3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.281714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69b7c7-bf15-49a4-9ffe-1bcc2bf9d3fb admin admin] PATH_INFO: `/v3/projects/af035ed3e74d474ba7163aaf6b742dc2/users/36cd5f720b2746ab9e76d9aa532cfd8c/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.304957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe69b7c7-bf15-49a4-9ffe-1bcc2bf9d3fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=af035ed3e74d474ba7163aaf6b742dc2, user_id=36cd5f720b2746ab9e76d9aa532cfd8c, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:42.310199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe69b7c7-bf15-49a4-9ffe-1bcc2bf9d3fb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:42.331638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 957/1898] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:42 2022] PUT /identity/v3/projects/af035ed3e74d474ba7163aaf6b742dc2/users/36cd5f720b2746ab9e76d9aa532cfd8c/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:42.334004 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c9e71-7c4b-43f6-8d40-e8f90e2caebc admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:42.335699 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3e7c9e71-7c4b-43f6-8d40-e8f90e2caebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:42.343940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82874324-774c-400d-a6d5-33e7ae885c5a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.344256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82874324-774c-400d-a6d5-33e7ae885c5a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.344533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82874324-774c-400d-a6d5-33e7ae885c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.354800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c9e71-7c4b-43f6-8d40-e8f90e2caebc admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:42.391479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-82874324-774c-400d-a6d5-33e7ae885c5a None None] MFA Rules not processed for user `36cd5f720b2746ab9e76d9aa532cfd8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:42.393259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-82874324-774c-400d-a6d5-33e7ae885c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:42.475244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 958/1899] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:42.479245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82874324-774c-400d-a6d5-33e7ae885c5a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:42.483410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:42.484357 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.484717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.485082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b admin admin] PATH_INFO: `/v3/projects/517a508ed697414ea0f66abc8f149988` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.487138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b admin admin] RBAC: Authorizing `identity:delete_project(project_id=517a508ed697414ea0f66abc8f149988)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:42.490898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:42.492097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82874324-774c-400d-a6d5-33e7ae885c5a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:42.499293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30623f0c-1853-4d2a-a364-794f11c74402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:42.499990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30623f0c-1853-4d2a-a364-794f11c74402 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.500389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30623f0c-1853-4d2a-a364-794f11c74402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.500714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30623f0c-1853-4d2a-a364-794f11c74402 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.501487 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30623f0c-1853-4d2a-a364-794f11c74402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:42.503774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30623f0c-1853-4d2a-a364-794f11c74402 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:42.513501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:42.528244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:42.559611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '517a508ed697414ea0f66abc8f149988', 'request_id': 'req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:42.562752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 959/1900] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:42 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:30:42.563707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30623f0c-1853-4d2a-a364-794f11c74402 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:42.572592 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '517a508ed697414ea0f66abc8f149988', 'request_id': 'req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:42.583080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '517a508ed697414ea0f66abc8f149988', 'request_id': 'req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:42.599989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b admin admin] The token cache is being invalidate because project 517a508ed697414ea0f66abc8f149988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:42.600578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:42.601320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a6185dd0-dfe6-4193-bc5b-7fd3fe89a93b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:42.603326 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 942/1901] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:42 2022] DELETE /identity/v3/projects/517a508ed697414ea0f66abc8f149988 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:42.609445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481b02-ef53-4f3a-8397-f1aac9f660ec None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.609644 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481b02-ef53-4f3a-8397-f1aac9f660ec None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.609892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481b02-ef53-4f3a-8397-f1aac9f660ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.635716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-19481b02-ef53-4f3a-8397-f1aac9f660ec None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:42.636892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-19481b02-ef53-4f3a-8397-f1aac9f660ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:42.664053 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30623f0c-1853-4d2a-a364-794f11c74402 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:42.668904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31955d2b-76c0-4478-90ba-d02dfb182c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:42.669522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31955d2b-76c0-4478-90ba-d02dfb182c43 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.670411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31955d2b-76c0-4478-90ba-d02dfb182c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.670688 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31955d2b-76c0-4478-90ba-d02dfb182c43 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.671322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31955d2b-76c0-4478-90ba-d02dfb182c43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:42.673301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31955d2b-76c0-4478-90ba-d02dfb182c43 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:42.696428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 960/1902] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:42 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:42.700423 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31955d2b-76c0-4478-90ba-d02dfb182c43 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:42.702976 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-31955d2b-76c0-4478-90ba-d02dfb182c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:42.705744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 943/1903] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:42.713385 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7153c-e068-469f-95b6-66961cc60b2f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.713679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7153c-e068-469f-95b6-66961cc60b2f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.713941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7153c-e068-469f-95b6-66961cc60b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.717743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31955d2b-76c0-4478-90ba-d02dfb182c43 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:42.722738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3eb30-c147-4935-b098-5f0f52a183f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:42.723358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3eb30-c147-4935-b098-5f0f52a183f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.723640 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3eb30-c147-4935-b098-5f0f52a183f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.723912 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3eb30-c147-4935-b098-5f0f52a183f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.724588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3eb30-c147-4935-b098-5f0f52a183f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:42.726364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3eb30-c147-4935-b098-5f0f52a183f6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:42.735741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f2c7153c-e068-469f-95b6-66961cc60b2f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:42.737205 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 961/1904] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:42.737340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f2c7153c-e068-469f-95b6-66961cc60b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:42.740854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3eb30-c147-4935-b098-5f0f52a183f6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:42.745021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 944/1905] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:42.749319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7153c-e068-469f-95b6-66961cc60b2f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:42.751205 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3eb30-c147-4935-b098-5f0f52a183f6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:42.756028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4a98f-500d-455c-a3b4-f5b2f4fb5a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:42.756728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4a98f-500d-455c-a3b4-f5b2f4fb5a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.757033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4a98f-500d-455c-a3b4-f5b2f4fb5a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.757301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4a98f-500d-455c-a3b4-f5b2f4fb5a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.757930 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4a98f-500d-455c-a3b4-f5b2f4fb5a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:42.758811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7153c-e068-469f-95b6-66961cc60b2f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:42.759835 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4a98f-500d-455c-a3b4-f5b2f4fb5a82 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:42.764093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf44f3a-e43d-43d1-8f32-297c3eca2ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:42.764687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf44f3a-e43d-43d1-8f32-297c3eca2ede admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.764965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf44f3a-e43d-43d1-8f32-297c3eca2ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.765241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf44f3a-e43d-43d1-8f32-297c3eca2ede admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.765872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf44f3a-e43d-43d1-8f32-297c3eca2ede admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:30:42.766168 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf44f3a-e43d-43d1-8f32-297c3eca2ede admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:42.767747 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf44f3a-e43d-43d1-8f32-297c3eca2ede admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:42.771205 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 962/1906] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:42.775463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4a98f-500d-455c-a3b4-f5b2f4fb5a82 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:42.781333 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 945/1907] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:30:42.785344 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf44f3a-e43d-43d1-8f32-297c3eca2ede admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:42.785477 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4a98f-500d-455c-a3b4-f5b2f4fb5a82 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:42.792925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4130bf-74e5-4490-84a5-f2bc9e52cc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:42.792925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4130bf-74e5-4490-84a5-f2bc9e52cc79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.792925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4130bf-74e5-4490-84a5-f2bc9e52cc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.792925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4130bf-74e5-4490-84a5-f2bc9e52cc79 admin admin] PATH_INFO: `/v3/projects/9abb1d2c44a24d6b81a51f048f2a4937/users/1e76e74407fa4330a9d0c0a3cba7106a/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.802003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf44f3a-e43d-43d1-8f32-297c3eca2ede admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:42.806816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f1bc5-1878-43d1-9b2e-a1336b03adf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:42.807466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f1bc5-1878-43d1-9b2e-a1336b03adf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.807698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f1bc5-1878-43d1-9b2e-a1336b03adf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.807972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f1bc5-1878-43d1-9b2e-a1336b03adf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.808649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f1bc5-1878-43d1-9b2e-a1336b03adf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:42.811594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f1bc5-1878-43d1-9b2e-a1336b03adf4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:42.814201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4130bf-74e5-4490-84a5-f2bc9e52cc79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9abb1d2c44a24d6b81a51f048f2a4937, user_id=1e76e74407fa4330a9d0c0a3cba7106a, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:42.816876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4130bf-74e5-4490-84a5-f2bc9e52cc79 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:42.831926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 963/1908] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:42 2022] PUT /identity/v3/projects/9abb1d2c44a24d6b81a51f048f2a4937/users/1e76e74407fa4330a9d0c0a3cba7106a/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:42.839901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 946/1909] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:42 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:30:42.844124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4130bf-74e5-4490-84a5-f2bc9e52cc79 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:42.845587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e242a6d-0787-4c25-9f1d-87018ae46fcb None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.845871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e242a6d-0787-4c25-9f1d-87018ae46fcb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.846146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e242a6d-0787-4c25-9f1d-87018ae46fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.881448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4e242a6d-0787-4c25-9f1d-87018ae46fcb None None] MFA Rules not processed for user `1e76e74407fa4330a9d0c0a3cba7106a`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:42.882755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4e242a6d-0787-4c25-9f1d-87018ae46fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:42.909943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4130bf-74e5-4490-84a5-f2bc9e52cc79 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:42.946295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:42.953570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:42.963942 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 947/1910] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:42.965589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e242a6d-0787-4c25-9f1d-87018ae46fcb None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:42.974062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5563d-243f-4f01-a34e-0d86d37afa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:42.974846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5563d-243f-4f01-a34e-0d86d37afa5e admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:42.975127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5563d-243f-4f01-a34e-0d86d37afa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:42.975394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5563d-243f-4f01-a34e-0d86d37afa5e admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:42.976059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5563d-243f-4f01-a34e-0d86d37afa5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:42.977981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5563d-243f-4f01-a34e-0d86d37afa5e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:43.000902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 964/1911] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:42 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:30:43.005370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5563d-243f-4f01-a34e-0d86d37afa5e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:43.017046 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5563d-243f-4f01-a34e-0d86d37afa5e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:43.022581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112cf0e-6aac-40ae-8697-3bb6d62dd3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:43.023314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112cf0e-6aac-40ae-8697-3bb6d62dd3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.023599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112cf0e-6aac-40ae-8697-3bb6d62dd3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.023868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112cf0e-6aac-40ae-8697-3bb6d62dd3e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.024551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112cf0e-6aac-40ae-8697-3bb6d62dd3e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:43.026149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112cf0e-6aac-40ae-8697-3bb6d62dd3e4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:43.035617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e242a6d-0787-4c25-9f1d-87018ae46fcb None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:43.036570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 965/1912] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:43.040557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112cf0e-6aac-40ae-8697-3bb6d62dd3e4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:43.043475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ed32e-141b-472d-8ea9-58c4e084e0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:43.044113 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ed32e-141b-472d-8ea9-58c4e084e0ce admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.044395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ed32e-141b-472d-8ea9-58c4e084e0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.044669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ed32e-141b-472d-8ea9-58c4e084e0ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.045391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ed32e-141b-472d-8ea9-58c4e084e0ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:43.047300 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ed32e-141b-472d-8ea9-58c4e084e0ce admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:43.050090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112cf0e-6aac-40ae-8697-3bb6d62dd3e4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:43.054655 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c1b80-97fa-4555-b40e-d58038b94c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:43.055233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c1b80-97fa-4555-b40e-d58038b94c99 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.055514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c1b80-97fa-4555-b40e-d58038b94c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.055785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c1b80-97fa-4555-b40e-d58038b94c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.056450 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c1b80-97fa-4555-b40e-d58038b94c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:43.058040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c1b80-97fa-4555-b40e-d58038b94c99 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:43.074862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 966/1913] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:43.079859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c1b80-97fa-4555-b40e-d58038b94c99 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:43.093426 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c1b80-97fa-4555-b40e-d58038b94c99 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:43.100348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 948/1914] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:42 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:43.100629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53b859-0cd7-4765-a4d8-15abc8db6c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:43.100629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53b859-0cd7-4765-a4d8-15abc8db6c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.100629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53b859-0cd7-4765-a4d8-15abc8db6c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.100629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53b859-0cd7-4765-a4d8-15abc8db6c5d admin admin] PATH_INFO: `/v3/projects/1114ae81a4604de99532a2d35cb429e4/users/a5665ba3aafb4b3f93f964cce62edb8e/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.108335 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ed32e-141b-472d-8ea9-58c4e084e0ce admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:43.129449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53b859-0cd7-4765-a4d8-15abc8db6c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1114ae81a4604de99532a2d35cb429e4, user_id=a5665ba3aafb4b3f93f964cce62edb8e, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:43.132586 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53b859-0cd7-4765-a4d8-15abc8db6c5d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:43.161103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 967/1915] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:43 2022] PUT /identity/v3/projects/1114ae81a4604de99532a2d35cb429e4/users/a5665ba3aafb4b3f93f964cce62edb8e/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:43.174368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7d17c-baa2-458c-89d6-669e56b7b42f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.174665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7d17c-baa2-458c-89d6-669e56b7b42f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.174947 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7d17c-baa2-458c-89d6-669e56b7b42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.217774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-dfb7d17c-baa2-458c-89d6-669e56b7b42f None None] MFA Rules not processed for user `a5665ba3aafb4b3f93f964cce62edb8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:43.219154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dfb7d17c-baa2-458c-89d6-669e56b7b42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:43.228025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ed32e-141b-472d-8ea9-58c4e084e0ce admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:43.323363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d34c5a-d2b9-4173-addb-256796a63932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:43.324051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d34c5a-d2b9-4173-addb-256796a63932 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.324230 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 968/1916] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:43.324331 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d34c5a-d2b9-4173-addb-256796a63932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.324599 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d34c5a-d2b9-4173-addb-256796a63932 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.325220 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d34c5a-d2b9-4173-addb-256796a63932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:43.328228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d34c5a-d2b9-4173-addb-256796a63932 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:43.332284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7d17c-baa2-458c-89d6-669e56b7b42f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:43.333621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dfb7d17c-baa2-458c-89d6-669e56b7b42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:43.353770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 949/1917] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:43 2022] POST /identity/v3/users => generated 299 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:43.354873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d34c5a-d2b9-4173-addb-256796a63932 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:43.364955 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d34c5a-d2b9-4173-addb-256796a63932 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:43.369661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee3ca8-64f3-4944-8470-1a3aefed7334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:43.370349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee3ca8-64f3-4944-8470-1a3aefed7334 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.370633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee3ca8-64f3-4944-8470-1a3aefed7334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.370904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee3ca8-64f3-4944-8470-1a3aefed7334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.371589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee3ca8-64f3-4944-8470-1a3aefed7334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:43.373386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee3ca8-64f3-4944-8470-1a3aefed7334 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:43.384929 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 950/1918] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:43.389580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee3ca8-64f3-4944-8470-1a3aefed7334 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:43.395701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:43.398419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7d17c-baa2-458c-89d6-669e56b7b42f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:43.399670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee3ca8-64f3-4944-8470-1a3aefed7334 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:43.402185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108762e-a6a7-49b7-aadf-862671852fd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:43.402829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108762e-a6a7-49b7-aadf-862671852fd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.403117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108762e-a6a7-49b7-aadf-862671852fd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.403386 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108762e-a6a7-49b7-aadf-862671852fd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.405584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fa8a6-80b9-4f73-a30c-3e56f4321cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:43.406389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fa8a6-80b9-4f73-a30c-3e56f4321cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.406726 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fa8a6-80b9-4f73-a30c-3e56f4321cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.407008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fa8a6-80b9-4f73-a30c-3e56f4321cbb admin admin] PATH_INFO: `/v3/projects/b4357fec76ed44cd98ca24da64c6e5c5/users/746ec25646b5440ab513c380c89816cc/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.412651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108762e-a6a7-49b7-aadf-862671852fd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:43.414130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108762e-a6a7-49b7-aadf-862671852fd9 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:43.423073 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fa8a6-80b9-4f73-a30c-3e56f4321cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4357fec76ed44cd98ca24da64c6e5c5, user_id=746ec25646b5440ab513c380c89816cc, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:43.423979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 969/1919] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:30:43.425928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fa8a6-80b9-4f73-a30c-3e56f4321cbb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:43.440605 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 951/1920] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:43 2022] PUT /identity/v3/projects/b4357fec76ed44cd98ca24da64c6e5c5/users/746ec25646b5440ab513c380c89816cc/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:43.444854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108762e-a6a7-49b7-aadf-862671852fd9 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:43.447578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fa8a6-80b9-4f73-a30c-3e56f4321cbb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:43.460820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108762e-a6a7-49b7-aadf-862671852fd9 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:43.475444 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:43.519822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fa8a6-80b9-4f73-a30c-3e56f4321cbb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:43.524901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f9658-8086-4833-b398-272489cd546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:43.525535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f9658-8086-4833-b398-272489cd546f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.525819 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f9658-8086-4833-b398-272489cd546f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.526094 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f9658-8086-4833-b398-272489cd546f admin admin] PATH_INFO: `/v3/users/a5665ba3aafb4b3f93f964cce62edb8e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.527430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f9658-8086-4833-b398-272489cd546f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5665ba3aafb4b3f93f964cce62edb8e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:43.530467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f9658-8086-4833-b398-272489cd546f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:43.535565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1ede3-b814-48e2-ace7-8ef369a74a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:43.537400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1ede3-b814-48e2-ace7-8ef369a74a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.537400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1ede3-b814-48e2-ace7-8ef369a74a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.537400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1ede3-b814-48e2-ace7-8ef369a74a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.537916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1ede3-b814-48e2-ace7-8ef369a74a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:43.540016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1ede3-b814-48e2-ace7-8ef369a74a82 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:43.573045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 970/1921] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:43.577741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1ede3-b814-48e2-ace7-8ef369a74a82 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:43.589185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1ede3-b814-48e2-ace7-8ef369a74a82 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:43.591258 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b83f9658-8086-4833-b398-272489cd546f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5665ba3aafb4b3f93f964cce62edb8e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:43.595315 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b74576-1a2d-41da-91ad-c4929c19c6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:43.596112 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b74576-1a2d-41da-91ad-c4929c19c6d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.596482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b74576-1a2d-41da-91ad-c4929c19c6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.596877 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b74576-1a2d-41da-91ad-c4929c19c6d7 admin admin] PATH_INFO: `/v3/projects/b4357fec76ed44cd98ca24da64c6e5c5/users/746ec25646b5440ab513c380c89816cc/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.599769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b74576-1a2d-41da-91ad-c4929c19c6d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4357fec76ed44cd98ca24da64c6e5c5, user_id=746ec25646b5440ab513c380c89816cc, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:43.600162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b83f9658-8086-4833-b398-272489cd546f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5665ba3aafb4b3f93f964cce62edb8e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:43.602790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b74576-1a2d-41da-91ad-c4929c19c6d7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:43.614648 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 971/1922] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:43 2022] PUT /identity/v3/projects/b4357fec76ed44cd98ca24da64c6e5c5/users/746ec25646b5440ab513c380c89816cc/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:43.627290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8c747-deb4-40f3-8245-94df3a505cd5 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.627692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8c747-deb4-40f3-8245-94df3a505cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.628715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8c747-deb4-40f3-8245-94df3a505cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.631422 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b83f9658-8086-4833-b398-272489cd546f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5665ba3aafb4b3f93f964cce62edb8e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:43.632999 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b83f9658-8086-4833-b398-272489cd546f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5665ba3aafb4b3f93f964cce62edb8e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:43.633968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b83f9658-8086-4833-b398-272489cd546f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5665ba3aafb4b3f93f964cce62edb8e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:43.663024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 952/1923] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:43 2022] DELETE /identity/v3/users/a5665ba3aafb4b3f93f964cce62edb8e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:43.664748 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dddb8-a64a-49fa-a89e-41077f5451ab None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.665223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dddb8-a64a-49fa-a89e-41077f5451ab None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.665710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dddb8-a64a-49fa-a89e-41077f5451ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.671568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a3c8c747-deb4-40f3-8245-94df3a505cd5 None None] MFA Rules not processed for user `746ec25646b5440ab513c380c89816cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:43.672829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a3c8c747-deb4-40f3-8245-94df3a505cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:43.699425 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-746dddb8-a64a-49fa-a89e-41077f5451ab None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:43.700735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-746dddb8-a64a-49fa-a89e-41077f5451ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:43.820284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:43.821777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 972/1924] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:43.839304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8c747-deb4-40f3-8245-94df3a505cd5 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:43.842484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a3c8c747-deb4-40f3-8245-94df3a505cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:43.850792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 953/1925] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:43.914342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dddb8-a64a-49fa-a89e-41077f5451ab None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:43.921985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-746dddb8-a64a-49fa-a89e-41077f5451ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:43.950299 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8c747-deb4-40f3-8245-94df3a505cd5 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:43.956870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714d7a3-42c1-4d36-b740-4853ef9bdda5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:43.957469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714d7a3-42c1-4d36-b740-4853ef9bdda5 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:43.957753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714d7a3-42c1-4d36-b740-4853ef9bdda5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:43.958213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714d7a3-42c1-4d36-b740-4853ef9bdda5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:43.960967 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d714d7a3-42c1-4d36-b740-4853ef9bdda5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:44.013047 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dddb8-a64a-49fa-a89e-41077f5451ab None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:44.017285 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c969e-b318-4b35-aa39-3397331e19f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:44.018504 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c969e-b318-4b35-aa39-3397331e19f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:44.018799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c969e-b318-4b35-aa39-3397331e19f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:44.019078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c969e-b318-4b35-aa39-3397331e19f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:44.037705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:44.044111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c969e-b318-4b35-aa39-3397331e19f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:44.047015 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c969e-b318-4b35-aa39-3397331e19f0 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:44.063850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 954/1926] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:44.068911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714d7a3-42c1-4d36-b740-4853ef9bdda5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:44.070491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714d7a3-42c1-4d36-b740-4853ef9bdda5 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:44.082469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 973/1927] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:44.696531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c969e-b318-4b35-aa39-3397331e19f0 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:44.698675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ef4c969e-b318-4b35-aa39-3397331e19f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:44.727425 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c969e-b318-4b35-aa39-3397331e19f0 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:44.745702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf0696-6e98-42d1-b676-4549374967b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:44.746450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf0696-6e98-42d1-b676-4549374967b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:44.747281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf0696-6e98-42d1-b676-4549374967b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:44.747647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf0696-6e98-42d1-b676-4549374967b5 admin admin] PATH_INFO: `/v3/projects/1114ae81a4604de99532a2d35cb429e4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:44.748933 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf0696-6e98-42d1-b676-4549374967b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1114ae81a4604de99532a2d35cb429e4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:44.750923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf0696-6e98-42d1-b676-4549374967b5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:44.833308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-15bf0696-6e98-42d1-b676-4549374967b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1114ae81a4604de99532a2d35cb429e4', 'request_id': 'req-15bf0696-6e98-42d1-b676-4549374967b5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:44.843641 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-15bf0696-6e98-42d1-b676-4549374967b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1114ae81a4604de99532a2d35cb429e4', 'request_id': 'req-15bf0696-6e98-42d1-b676-4549374967b5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:44.852764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-15bf0696-6e98-42d1-b676-4549374967b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1114ae81a4604de99532a2d35cb429e4', 'request_id': 'req-15bf0696-6e98-42d1-b676-4549374967b5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:44.880402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-15bf0696-6e98-42d1-b676-4549374967b5 admin admin] The token cache is being invalidate because project 1114ae81a4604de99532a2d35cb429e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:44.880809 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-15bf0696-6e98-42d1-b676-4549374967b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:44.881531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-15bf0696-6e98-42d1-b676-4549374967b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:44.883435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 955/1928] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:44 2022] DELETE /identity/v3/projects/1114ae81a4604de99532a2d35cb429e4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:44.888291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d64b45-2461-4543-b5ed-e3d02f4a9e28 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:44.889290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d64b45-2461-4543-b5ed-e3d02f4a9e28 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:44.889370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d64b45-2461-4543-b5ed-e3d02f4a9e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:44.911753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-00d64b45-2461-4543-b5ed-e3d02f4a9e28 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:44.913009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-00d64b45-2461-4543-b5ed-e3d02f4a9e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:44.975091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 974/1929] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:44.983474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8c463-b1b0-4762-b193-2a7d97c644dc None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:44.983842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8c463-b1b0-4762-b193-2a7d97c644dc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:44.984159 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8c463-b1b0-4762-b193-2a7d97c644dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:45.004312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-efd8c463-b1b0-4762-b193-2a7d97c644dc None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:45.005668 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-efd8c463-b1b0-4762-b193-2a7d97c644dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:45.014526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 956/1930] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:45.018931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d64b45-2461-4543-b5ed-e3d02f4a9e28 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:45.029378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d64b45-2461-4543-b5ed-e3d02f4a9e28 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:45.035387 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8819c94-2b91-4e87-a121-8a349709b940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:45.036283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8819c94-2b91-4e87-a121-8a349709b940 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:45.036619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8819c94-2b91-4e87-a121-8a349709b940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:45.037019 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8819c94-2b91-4e87-a121-8a349709b940 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:45.037834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8819c94-2b91-4e87-a121-8a349709b940 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:30:45.038216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8819c94-2b91-4e87-a121-8a349709b940 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:45.040263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8819c94-2b91-4e87-a121-8a349709b940 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:45.054144 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 975/1931] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:30:45.058596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8c463-b1b0-4762-b193-2a7d97c644dc None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:45.068810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8c463-b1b0-4762-b193-2a7d97c644dc None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:45.073656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa4aa7-2983-42b8-a9aa-e3e9baff46fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:45.074390 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa4aa7-2983-42b8-a9aa-e3e9baff46fd admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:45.074778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa4aa7-2983-42b8-a9aa-e3e9baff46fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:45.075239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa4aa7-2983-42b8-a9aa-e3e9baff46fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:45.076021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa4aa7-2983-42b8-a9aa-e3e9baff46fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:45.077926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa4aa7-2983-42b8-a9aa-e3e9baff46fd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:45.116561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 957/1932] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:45 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:30:45.117986 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8819c94-2b91-4e87-a121-8a349709b940 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:45.130001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8819c94-2b91-4e87-a121-8a349709b940 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:45.187587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc96d8-3f05-4bff-86b4-1cc2aeaccd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:45.188229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc96d8-3f05-4bff-86b4-1cc2aeaccd9a admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:45.188505 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc96d8-3f05-4bff-86b4-1cc2aeaccd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:45.188808 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc96d8-3f05-4bff-86b4-1cc2aeaccd9a admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:45.189448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc96d8-3f05-4bff-86b4-1cc2aeaccd9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:45.191829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc96d8-3f05-4bff-86b4-1cc2aeaccd9a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:45.219532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 976/1933] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:45 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:45.223890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa4aa7-2983-42b8-a9aa-e3e9baff46fd admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:45.235545 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa4aa7-2983-42b8-a9aa-e3e9baff46fd admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:45.240684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c926c-2529-4516-871c-e6b15bc022e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:45.241312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c926c-2529-4516-871c-e6b15bc022e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:45.241626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c926c-2529-4516-871c-e6b15bc022e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:45.241901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c926c-2529-4516-871c-e6b15bc022e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:45.242555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c926c-2529-4516-871c-e6b15bc022e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:45.244269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c926c-2529-4516-871c-e6b15bc022e9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:45.265109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 958/1934] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:45.266925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc96d8-3f05-4bff-86b4-1cc2aeaccd9a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:45.278176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc96d8-3f05-4bff-86b4-1cc2aeaccd9a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:45.283578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200514a-0d7c-4143-81a3-659a063dcb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:45.284220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200514a-0d7c-4143-81a3-659a063dcb03 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:45.284501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200514a-0d7c-4143-81a3-659a063dcb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:45.284772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200514a-0d7c-4143-81a3-659a063dcb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:45.285416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200514a-0d7c-4143-81a3-659a063dcb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:45.287004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200514a-0d7c-4143-81a3-659a063dcb03 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:45.298430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 977/1935] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:45.302637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c926c-2529-4516-871c-e6b15bc022e9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:45.312793 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c926c-2529-4516-871c-e6b15bc022e9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:45.317344 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d83b9d-3737-4702-b357-dcf3130df2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:45.317986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d83b9d-3737-4702-b357-dcf3130df2ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:45.318267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d83b9d-3737-4702-b357-dcf3130df2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:45.318542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d83b9d-3737-4702-b357-dcf3130df2ab admin admin] PATH_INFO: `/v3/projects/59ceab33070f47babe3ae56fe06799bd/users/2f73a9c674e44abda0cbe45881a9875e/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:45.333715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d83b9d-3737-4702-b357-dcf3130df2ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ceab33070f47babe3ae56fe06799bd, user_id=2f73a9c674e44abda0cbe45881a9875e, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:45.336451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d83b9d-3737-4702-b357-dcf3130df2ab admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:45.349510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 959/1936] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:45 2022] PUT /identity/v3/projects/59ceab33070f47babe3ae56fe06799bd/users/2f73a9c674e44abda0cbe45881a9875e/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:45.360384 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007f507-ab0f-4418-bd61-69c150a7417c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:45.360672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007f507-ab0f-4418-bd61-69c150a7417c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:45.360945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007f507-ab0f-4418-bd61-69c150a7417c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:45.389327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4007f507-ab0f-4418-bd61-69c150a7417c None None] MFA Rules not processed for user `2f73a9c674e44abda0cbe45881a9875e`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:45.390608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4007f507-ab0f-4418-bd61-69c150a7417c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:45.424315 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:45.460990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 978/1937] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:45.472606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d83b9d-3737-4702-b357-dcf3130df2ab admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:45.474282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c2d83b9d-3737-4702-b357-dcf3130df2ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:45.500927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:45.538722 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d83b9d-3737-4702-b357-dcf3130df2ab admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:45.543325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b910fc9-aa68-4a17-87c8-82fb0e57fe5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:45.543945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b910fc9-aa68-4a17-87c8-82fb0e57fe5d service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:45.544260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b910fc9-aa68-4a17-87c8-82fb0e57fe5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:45.544561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b910fc9-aa68-4a17-87c8-82fb0e57fe5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:45.556301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b910fc9-aa68-4a17-87c8-82fb0e57fe5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:45.557748 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b910fc9-aa68-4a17-87c8-82fb0e57fe5d service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:45.579317 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 960/1938] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:45.787710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007f507-ab0f-4418-bd61-69c150a7417c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:45.789901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4007f507-ab0f-4418-bd61-69c150a7417c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:45.804681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007f507-ab0f-4418-bd61-69c150a7417c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:45.809607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b8feb-3973-4860-ba54-055aa51681f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:45.810344 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b8feb-3973-4860-ba54-055aa51681f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:45.810771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b8feb-3973-4860-ba54-055aa51681f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:45.811467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b8feb-3973-4860-ba54-055aa51681f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:45.813289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2a9b8feb-3973-4860-ba54-055aa51681f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:45.912136 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b8feb-3973-4860-ba54-055aa51681f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:45.914288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b8feb-3973-4860-ba54-055aa51681f2 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:45.928280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 979/1939] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:46.611957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b910fc9-aa68-4a17-87c8-82fb0e57fe5d service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:46.620618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b910fc9-aa68-4a17-87c8-82fb0e57fe5d service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:46.624437 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d03935-1d2c-4517-835d-e50018d890a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:46.625030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d03935-1d2c-4517-835d-e50018d890a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:46.625372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d03935-1d2c-4517-835d-e50018d890a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:46.625651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d03935-1d2c-4517-835d-e50018d890a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:46.627345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-14d03935-1d2c-4517-835d-e50018d890a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:46.698992 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d03935-1d2c-4517-835d-e50018d890a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:46.700468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d03935-1d2c-4517-835d-e50018d890a4 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:46.713170 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 961/1940] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:48.831386 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b8feb-3973-4860-ba54-055aa51681f2 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:48.988290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b8feb-3973-4860-ba54-055aa51681f2 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:48.993703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:48.994452 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:48.994738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:48.995012 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] PATH_INFO: `/v3/users/2f73a9c674e44abda0cbe45881a9875e` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:48.996263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f73a9c674e44abda0cbe45881a9875e)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:48.999032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:49.058452 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f73a9c674e44abda0cbe45881a9875e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:49.059951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f73a9c674e44abda0cbe45881a9875e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:49.070713 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f73a9c674e44abda0cbe45881a9875e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:49.079428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f73a9c674e44abda0cbe45881a9875e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:49.088656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:49.094555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f73a9c674e44abda0cbe45881a9875e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:49.102618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 980/1941] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:48 2022] DELETE /identity/v3/users/2f73a9c674e44abda0cbe45881a9875e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:49.107174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace41cea-1db4-4692-a02d-1bec46129880 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:49.107688 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace41cea-1db4-4692-a02d-1bec46129880 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:49.107781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace41cea-1db4-4692-a02d-1bec46129880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:49.128313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ace41cea-1db4-4692-a02d-1bec46129880 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:49.129543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ace41cea-1db4-4692-a02d-1bec46129880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:49.193302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 962/1942] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:49.203786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:49.205218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:49.264640 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6544c-e20b-44bb-800c-1d66a14c8a03 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:49.268426 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa4ba4-84c9-41f5-8871-0fbbf6b9a6d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:49.269048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4ba4-84c9-41f5-8871-0fbbf6b9a6d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:49.269331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4ba4-84c9-41f5-8871-0fbbf6b9a6d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:49.269607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4ba4-84c9-41f5-8871-0fbbf6b9a6d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:49.278926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa4ba4-84c9-41f5-8871-0fbbf6b9a6d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:49.280804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa4ba4-84c9-41f5-8871-0fbbf6b9a6d3 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:49.290350 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 981/1943] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:49.780496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace41cea-1db4-4692-a02d-1bec46129880 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:49.781914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ace41cea-1db4-4692-a02d-1bec46129880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:49.808761 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace41cea-1db4-4692-a02d-1bec46129880 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:49.821701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6166e8-468f-4683-a4e6-d555261961f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:49.822518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6166e8-468f-4683-a4e6-d555261961f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:49.823617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6166e8-468f-4683-a4e6-d555261961f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:49.824039 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6166e8-468f-4683-a4e6-d555261961f8 admin admin] PATH_INFO: `/v3/projects/59ceab33070f47babe3ae56fe06799bd` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:49.825353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6166e8-468f-4683-a4e6-d555261961f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59ceab33070f47babe3ae56fe06799bd)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:49.828449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6166e8-468f-4683-a4e6-d555261961f8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:49.878961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ea6166e8-468f-4683-a4e6-d555261961f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59ceab33070f47babe3ae56fe06799bd', 'request_id': 'req-ea6166e8-468f-4683-a4e6-d555261961f8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:49.886282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:49.892945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ea6166e8-468f-4683-a4e6-d555261961f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59ceab33070f47babe3ae56fe06799bd', 'request_id': 'req-ea6166e8-468f-4683-a4e6-d555261961f8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:49.903716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ea6166e8-468f-4683-a4e6-d555261961f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59ceab33070f47babe3ae56fe06799bd', 'request_id': 'req-ea6166e8-468f-4683-a4e6-d555261961f8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:49.916910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ea6166e8-468f-4683-a4e6-d555261961f8 admin admin] The token cache is being invalidate because project 59ceab33070f47babe3ae56fe06799bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:49.917290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ea6166e8-468f-4683-a4e6-d555261961f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:49.918040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ea6166e8-468f-4683-a4e6-d555261961f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:49.923771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 963/1944] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:49 2022] DELETE /identity/v3/projects/59ceab33070f47babe3ae56fe06799bd => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:49.925420 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d4748-cb10-4eb0-8cbb-467fb2d6e813 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:49.925894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d4748-cb10-4eb0-8cbb-467fb2d6e813 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:49.926003 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d4748-cb10-4eb0-8cbb-467fb2d6e813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:49.957649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a38d4748-cb10-4eb0-8cbb-467fb2d6e813 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:49.959445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a38d4748-cb10-4eb0-8cbb-467fb2d6e813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:50.053000 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 982/1945] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:50.062941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab006d-0aa2-4b06-a612-950e95e1ffe0 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:50.063191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab006d-0aa2-4b06-a612-950e95e1ffe0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:50.063389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab006d-0aa2-4b06-a612-950e95e1ffe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:50.098858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-55ab006d-0aa2-4b06-a612-950e95e1ffe0 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:50.100277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-55ab006d-0aa2-4b06-a612-950e95e1ffe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:50.111089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 964/1946] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:30:50.116301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d4748-cb10-4eb0-8cbb-467fb2d6e813 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:50.131215 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d4748-cb10-4eb0-8cbb-467fb2d6e813 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:50.144674 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c88192-c9d9-4a5f-921f-bcfb45c14167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:50.145360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c88192-c9d9-4a5f-921f-bcfb45c14167 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:50.145641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c88192-c9d9-4a5f-921f-bcfb45c14167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:50.145916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c88192-c9d9-4a5f-921f-bcfb45c14167 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:50.146574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c88192-c9d9-4a5f-921f-bcfb45c14167 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:30:50.147227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c88192-c9d9-4a5f-921f-bcfb45c14167 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:50.148534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c88192-c9d9-4a5f-921f-bcfb45c14167 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:50.162487 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 983/1947] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:30:50.167020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab006d-0aa2-4b06-a612-950e95e1ffe0 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:50.176504 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab006d-0aa2-4b06-a612-950e95e1ffe0 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:50.181149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3e368-c009-41f5-9159-1035b41afcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:50.181750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3e368-c009-41f5-9159-1035b41afcdf admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:50.182031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3e368-c009-41f5-9159-1035b41afcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:50.182303 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3e368-c009-41f5-9159-1035b41afcdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:50.182933 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3e368-c009-41f5-9159-1035b41afcdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:50.184877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3e368-c009-41f5-9159-1035b41afcdf admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:50.218234 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 965/1948] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:50 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:50.219321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c88192-c9d9-4a5f-921f-bcfb45c14167 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:50.229462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c88192-c9d9-4a5f-921f-bcfb45c14167 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:50.313231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94464248-e79e-4a5a-9f5b-d3b23708e293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:50.313976 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94464248-e79e-4a5a-9f5b-d3b23708e293 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:50.314260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94464248-e79e-4a5a-9f5b-d3b23708e293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:50.314543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94464248-e79e-4a5a-9f5b-d3b23708e293 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:50.315203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94464248-e79e-4a5a-9f5b-d3b23708e293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:50.317187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94464248-e79e-4a5a-9f5b-d3b23708e293 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:50.340086 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 984/1949] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:50 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:30:50.345221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3e368-c009-41f5-9159-1035b41afcdf admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:50.357363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3e368-c009-41f5-9159-1035b41afcdf admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:50.380543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347da869-8fe9-4d66-b8dd-49a413e09058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:50.381491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347da869-8fe9-4d66-b8dd-49a413e09058 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:50.382072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347da869-8fe9-4d66-b8dd-49a413e09058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:50.382548 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347da869-8fe9-4d66-b8dd-49a413e09058 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:50.383649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347da869-8fe9-4d66-b8dd-49a413e09058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:50.385802 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347da869-8fe9-4d66-b8dd-49a413e09058 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:50.401375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 966/1950] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:50.405604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94464248-e79e-4a5a-9f5b-d3b23708e293 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:50.419808 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94464248-e79e-4a5a-9f5b-d3b23708e293 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:50.440763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc949155-6266-4d2f-a247-bf8792419757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:50.441436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc949155-6266-4d2f-a247-bf8792419757 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:50.441723 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc949155-6266-4d2f-a247-bf8792419757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:50.441996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc949155-6266-4d2f-a247-bf8792419757 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:50.442663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc949155-6266-4d2f-a247-bf8792419757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:50.459204 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc949155-6266-4d2f-a247-bf8792419757 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:50.466084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:50.484345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 985/1951] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:50.485324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347da869-8fe9-4d66-b8dd-49a413e09058 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:50.495323 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347da869-8fe9-4d66-b8dd-49a413e09058 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:50.501384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298992c-bef5-47b7-821e-07a0f02d5a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:50.502280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298992c-bef5-47b7-821e-07a0f02d5a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:50.502747 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298992c-bef5-47b7-821e-07a0f02d5a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:50.503166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298992c-bef5-47b7-821e-07a0f02d5a84 admin admin] PATH_INFO: `/v3/projects/86999cdbef3c41ff8b2a529f767108c4/users/cb46fd7dcdad4637a06a8af3653cf399/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:50.535375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298992c-bef5-47b7-821e-07a0f02d5a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86999cdbef3c41ff8b2a529f767108c4, user_id=cb46fd7dcdad4637a06a8af3653cf399, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:50.538924 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298992c-bef5-47b7-821e-07a0f02d5a84 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:50.553303 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 967/1952] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:50 2022] PUT /identity/v3/projects/86999cdbef3c41ff8b2a529f767108c4/users/cb46fd7dcdad4637a06a8af3653cf399/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:50.564472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93d575-cb74-4c87-b1aa-363f1fe4987b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:50.569042 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93d575-cb74-4c87-b1aa-363f1fe4987b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:50.569402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93d575-cb74-4c87-b1aa-363f1fe4987b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:50.634612 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1b93d575-cb74-4c87-b1aa-363f1fe4987b None None] MFA Rules not processed for user `cb46fd7dcdad4637a06a8af3653cf399`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:50.635906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1b93d575-cb74-4c87-b1aa-363f1fe4987b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:50.774457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 986/1953] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:50.778791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298992c-bef5-47b7-821e-07a0f02d5a84 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:50.870303 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298992c-bef5-47b7-821e-07a0f02d5a84 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:50.876741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28281d3c-7b68-45d2-a053-26f353b76ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:50.877512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28281d3c-7b68-45d2-a053-26f353b76ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:50.877860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28281d3c-7b68-45d2-a053-26f353b76ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:50.879705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28281d3c-7b68-45d2-a053-26f353b76ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:50.879705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28281d3c-7b68-45d2-a053-26f353b76ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:50.881276 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28281d3c-7b68-45d2-a053-26f353b76ff8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:50.922632 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 968/1954] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:50 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:50.926825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93d575-cb74-4c87-b1aa-363f1fe4987b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:50.938374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93d575-cb74-4c87-b1aa-363f1fe4987b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:51.006462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d420cd-fe48-4854-9f1e-21ccd7c2242a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:51.007161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d420cd-fe48-4854-9f1e-21ccd7c2242a admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:51.007516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d420cd-fe48-4854-9f1e-21ccd7c2242a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:51.007860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d420cd-fe48-4854-9f1e-21ccd7c2242a admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:51.008866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d420cd-fe48-4854-9f1e-21ccd7c2242a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:51.010906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d420cd-fe48-4854-9f1e-21ccd7c2242a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:51.043287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 987/1955] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:50 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:30:51.048897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28281d3c-7b68-45d2-a053-26f353b76ff8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:51.060834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28281d3c-7b68-45d2-a053-26f353b76ff8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:51.066424 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d588e52-5ea6-478a-9919-d20b28053d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:51.067232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d588e52-5ea6-478a-9919-d20b28053d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:51.067816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d588e52-5ea6-478a-9919-d20b28053d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:51.067816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d588e52-5ea6-478a-9919-d20b28053d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:51.068261 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d588e52-5ea6-478a-9919-d20b28053d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:51.069959 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d588e52-5ea6-478a-9919-d20b28053d19 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:51.083479 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 969/1956] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:51.087565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d420cd-fe48-4854-9f1e-21ccd7c2242a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:51.102438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d420cd-fe48-4854-9f1e-21ccd7c2242a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:51.110023 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76941a3f-8b8b-42a2-8a88-c915906f480b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:51.110820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76941a3f-8b8b-42a2-8a88-c915906f480b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:51.111171 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76941a3f-8b8b-42a2-8a88-c915906f480b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:51.111505 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76941a3f-8b8b-42a2-8a88-c915906f480b admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:51.112360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76941a3f-8b8b-42a2-8a88-c915906f480b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:51.114550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76941a3f-8b8b-42a2-8a88-c915906f480b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:51.130505 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 988/1957] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:51.131897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d588e52-5ea6-478a-9919-d20b28053d19 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:51.144100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d588e52-5ea6-478a-9919-d20b28053d19 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:51.149369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1393b-5d56-4028-ab70-fc3e6c2a5072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:51.150183 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1393b-5d56-4028-ab70-fc3e6c2a5072 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:51.150737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1393b-5d56-4028-ab70-fc3e6c2a5072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:51.151043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1393b-5d56-4028-ab70-fc3e6c2a5072 admin admin] PATH_INFO: `/v3/projects/3eb6148713864fa1a6ab8e45430927fe/users/f4edb28cb61b4b3484bf003d764e2bea/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:51.168149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1393b-5d56-4028-ab70-fc3e6c2a5072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb6148713864fa1a6ab8e45430927fe, user_id=f4edb28cb61b4b3484bf003d764e2bea, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:51.171592 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1393b-5d56-4028-ab70-fc3e6c2a5072 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:51.192726 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 970/1958] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:51 2022] PUT /identity/v3/projects/3eb6148713864fa1a6ab8e45430927fe/users/f4edb28cb61b4b3484bf003d764e2bea/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:51.204586 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466d13c-0bdb-4323-9cd2-0beb22a9c07f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:51.204870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466d13c-0bdb-4323-9cd2-0beb22a9c07f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:51.205097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466d13c-0bdb-4323-9cd2-0beb22a9c07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:51.240852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4466d13c-0bdb-4323-9cd2-0beb22a9c07f None None] MFA Rules not processed for user `f4edb28cb61b4b3484bf003d764e2bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:51.242265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4466d13c-0bdb-4323-9cd2-0beb22a9c07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:51.339339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:51.347513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 989/1959] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:51.351229 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1393b-5d56-4028-ab70-fc3e6c2a5072 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:51.390083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:51.426165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1393b-5d56-4028-ab70-fc3e6c2a5072 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:51.430850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be626b9f-6f26-4bc2-82d5-bcdb8c1d3c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:51.431491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be626b9f-6f26-4bc2-82d5-bcdb8c1d3c62 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:51.431768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be626b9f-6f26-4bc2-82d5-bcdb8c1d3c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:51.432071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be626b9f-6f26-4bc2-82d5-bcdb8c1d3c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:51.432698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be626b9f-6f26-4bc2-82d5-bcdb8c1d3c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:51.434595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be626b9f-6f26-4bc2-82d5-bcdb8c1d3c62 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:51.465329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 971/1960] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:51 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:51.469544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466d13c-0bdb-4323-9cd2-0beb22a9c07f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:51.482479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466d13c-0bdb-4323-9cd2-0beb22a9c07f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:51.547799 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cdd9c-6836-4ee1-8568-43e05796d124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:51.548762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cdd9c-6836-4ee1-8568-43e05796d124 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:51.549219 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cdd9c-6836-4ee1-8568-43e05796d124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:51.549670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cdd9c-6836-4ee1-8568-43e05796d124 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:51.550746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202cdd9c-6836-4ee1-8568-43e05796d124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:51.554195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202cdd9c-6836-4ee1-8568-43e05796d124 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:51.582504 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 990/1961] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:51 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:30:51.586298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be626b9f-6f26-4bc2-82d5-bcdb8c1d3c62 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:51.598290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be626b9f-6f26-4bc2-82d5-bcdb8c1d3c62 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:51.609377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda4fe2-fbc9-45b6-8b79-c8264cb95289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:51.610002 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda4fe2-fbc9-45b6-8b79-c8264cb95289 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:51.610286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda4fe2-fbc9-45b6-8b79-c8264cb95289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:51.610558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda4fe2-fbc9-45b6-8b79-c8264cb95289 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:51.611522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda4fe2-fbc9-45b6-8b79-c8264cb95289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:51.614255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda4fe2-fbc9-45b6-8b79-c8264cb95289 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:51.628110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 972/1962] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:30:51.632573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cdd9c-6836-4ee1-8568-43e05796d124 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:51.643259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cdd9c-6836-4ee1-8568-43e05796d124 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:51.648523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3826f9-d3d9-472c-a4bc-4d8796071ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:51.649192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3826f9-d3d9-472c-a4bc-4d8796071ceb admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:51.649472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3826f9-d3d9-472c-a4bc-4d8796071ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:51.649747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3826f9-d3d9-472c-a4bc-4d8796071ceb admin admin] PATH_INFO: `/v3/projects/1371be62105343e8a845703cfdb30980/users/ceb104f09a174a4f8fa8d4ebacb4037b/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:51.668044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3826f9-d3d9-472c-a4bc-4d8796071ceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1371be62105343e8a845703cfdb30980, user_id=ceb104f09a174a4f8fa8d4ebacb4037b, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:51.671694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3826f9-d3d9-472c-a4bc-4d8796071ceb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:51.692668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 991/1963] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:51 2022] PUT /identity/v3/projects/1371be62105343e8a845703cfdb30980/users/ceb104f09a174a4f8fa8d4ebacb4037b/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:51.697031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda4fe2-fbc9-45b6-8b79-c8264cb95289 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:51.707588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda4fe2-fbc9-45b6-8b79-c8264cb95289 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:51.772239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfa65c-07b7-4503-a166-e750da1a5907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:51.772849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfa65c-07b7-4503-a166-e750da1a5907 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:51.773131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfa65c-07b7-4503-a166-e750da1a5907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:51.773400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfa65c-07b7-4503-a166-e750da1a5907 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:51.774034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfa65c-07b7-4503-a166-e750da1a5907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:51.775647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfa65c-07b7-4503-a166-e750da1a5907 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:51.785533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 973/1964] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:51.789985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3826f9-d3d9-472c-a4bc-4d8796071ceb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:51.801467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3826f9-d3d9-472c-a4bc-4d8796071ceb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:51.806162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfb2dd-f244-47af-ba20-d2d1c10077b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:51.806968 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfb2dd-f244-47af-ba20-d2d1c10077b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:51.807244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfb2dd-f244-47af-ba20-d2d1c10077b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:51.807511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfb2dd-f244-47af-ba20-d2d1c10077b8 admin admin] PATH_INFO: `/v3/projects/1371be62105343e8a845703cfdb30980/users/ceb104f09a174a4f8fa8d4ebacb4037b/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:51.809535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfb2dd-f244-47af-ba20-d2d1c10077b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1371be62105343e8a845703cfdb30980, user_id=ceb104f09a174a4f8fa8d4ebacb4037b, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:51.813290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfb2dd-f244-47af-ba20-d2d1c10077b8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:51.828746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 992/1965] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:51 2022] PUT /identity/v3/projects/1371be62105343e8a845703cfdb30980/users/ceb104f09a174a4f8fa8d4ebacb4037b/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:51.842523 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e15aa-e3ee-4574-a788-c3958e0e4398 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:51.842982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e15aa-e3ee-4574-a788-c3958e0e4398 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:51.843296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e15aa-e3ee-4574-a788-c3958e0e4398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:51.874807 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-036e15aa-e3ee-4574-a788-c3958e0e4398 None None] MFA Rules not processed for user `ceb104f09a174a4f8fa8d4ebacb4037b`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:51.876096 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-036e15aa-e3ee-4574-a788-c3958e0e4398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:51.957608 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 974/1966] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:51.971750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfb2dd-f244-47af-ba20-d2d1c10077b8 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:51.973875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3bcfb2dd-f244-47af-ba20-d2d1c10077b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:51.986060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e15aa-e3ee-4574-a788-c3958e0e4398 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:51.987490 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-036e15aa-e3ee-4574-a788-c3958e0e4398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:52.017338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:52.038346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfb2dd-f244-47af-ba20-d2d1c10077b8 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:52.042029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70414c-2e94-49ce-b61f-74d2944a045a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:52.042651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70414c-2e94-49ce-b61f-74d2944a045a service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:52.042928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70414c-2e94-49ce-b61f-74d2944a045a service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:52.043202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70414c-2e94-49ce-b61f-74d2944a045a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:52.057097 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e15aa-e3ee-4574-a788-c3958e0e4398 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:52.061746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:52.062373 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:52.062652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:52.062922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] PATH_INFO: `/v3/users/36cd5f720b2746ab9e76d9aa532cfd8c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:52.064078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36cd5f720b2746ab9e76d9aa532cfd8c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:52.066023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:52.104130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70414c-2e94-49ce-b61f-74d2944a045a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:52.105575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70414c-2e94-49ce-b61f-74d2944a045a service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:52.115862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36cd5f720b2746ab9e76d9aa532cfd8c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:52.116220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 993/1967] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:52.128337 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36cd5f720b2746ab9e76d9aa532cfd8c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:52.141629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36cd5f720b2746ab9e76d9aa532cfd8c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:52.142709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36cd5f720b2746ab9e76d9aa532cfd8c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:52.143407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36cd5f720b2746ab9e76d9aa532cfd8c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:52.159452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 975/1968] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:51 2022] DELETE /identity/v3/users/36cd5f720b2746ab9e76d9aa532cfd8c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:30:52.164028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ee7e1-e4d5-4402-9bd5-a3b14e6e4643 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:52.164388 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ee7e1-e4d5-4402-9bd5-a3b14e6e4643 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:52.164657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ee7e1-e4d5-4402-9bd5-a3b14e6e4643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:52.174750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:52.189111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a16ee7e1-e4d5-4402-9bd5-a3b14e6e4643 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:52.190869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a16ee7e1-e4d5-4402-9bd5-a3b14e6e4643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:52.259865 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 994/1969] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:52.270753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:52.272335 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:52.349878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8d1b7-a9b3-4825-9be8-3a174c3ae0f4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:52.354853 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f924f-7f3d-4bf2-b5a4-d069cdf7588d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:52.355529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f924f-7f3d-4bf2-b5a4-d069cdf7588d service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:52.356640 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f924f-7f3d-4bf2-b5a4-d069cdf7588d service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:52.356974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f924f-7f3d-4bf2-b5a4-d069cdf7588d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:52.368547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f924f-7f3d-4bf2-b5a4-d069cdf7588d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:52.371140 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f924f-7f3d-4bf2-b5a4-d069cdf7588d service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:52.385280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 976/1970] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:52.966150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ee7e1-e4d5-4402-9bd5-a3b14e6e4643 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:52.968413 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a16ee7e1-e4d5-4402-9bd5-a3b14e6e4643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:52.981118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ee7e1-e4d5-4402-9bd5-a3b14e6e4643 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:52.986550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:52.987637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:52.987637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:52.987856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] PATH_INFO: `/v3/projects/af035ed3e74d474ba7163aaf6b742dc2` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:52.988990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=af035ed3e74d474ba7163aaf6b742dc2)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:52.990943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:53.042031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af035ed3e74d474ba7163aaf6b742dc2', 'request_id': 'req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:53.048412 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af035ed3e74d474ba7163aaf6b742dc2', 'request_id': 'req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:53.054702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af035ed3e74d474ba7163aaf6b742dc2', 'request_id': 'req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:53.070378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] The token cache is being invalidate because project af035ed3e74d474ba7163aaf6b742dc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:53.070588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:53.071865 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:53.073438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 995/1971] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:52 2022] DELETE /identity/v3/projects/af035ed3e74d474ba7163aaf6b742dc2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:53.078327 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f924f-7f3d-4bf2-b5a4-d069cdf7588d service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:53.088314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f924f-7f3d-4bf2-b5a4-d069cdf7588d service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:53.157702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:53.175085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:53.175859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:53.176261 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:53.176627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] PATH_INFO: `/v3/users/1e76e74407fa4330a9d0c0a3cba7106a` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:53.177829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e76e74407fa4330a9d0c0a3cba7106a)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:53.179764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:53.236349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e76e74407fa4330a9d0c0a3cba7106a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:53.244174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e76e74407fa4330a9d0c0a3cba7106a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:53.274493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e76e74407fa4330a9d0c0a3cba7106a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:53.275256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e76e74407fa4330a9d0c0a3cba7106a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:53.275912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e76e74407fa4330a9d0c0a3cba7106a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:53.315616 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 977/1972] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:53 2022] DELETE /identity/v3/users/1e76e74407fa4330a9d0c0a3cba7106a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:53.830218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:53.831782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:53.847746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fe15b-85c1-4fc9-8c55-d03e4523ce7d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:53.926594 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:53.927358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:53.927715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:53.928097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] PATH_INFO: `/v3/projects/9abb1d2c44a24d6b81a51f048f2a4937` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:53.929296 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9abb1d2c44a24d6b81a51f048f2a4937)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:53.931211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:53.966014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:53.982366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9abb1d2c44a24d6b81a51f048f2a4937', 'request_id': 'req-84d3eea5-5016-48cf-8d54-5c7e82758724'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:53.988482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9abb1d2c44a24d6b81a51f048f2a4937', 'request_id': 'req-84d3eea5-5016-48cf-8d54-5c7e82758724'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:53.994813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9abb1d2c44a24d6b81a51f048f2a4937', 'request_id': 'req-84d3eea5-5016-48cf-8d54-5c7e82758724'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:54.014298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] The token cache is being invalidate because project 9abb1d2c44a24d6b81a51f048f2a4937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:54.014598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:54.015254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:54.016791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 996/1973] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:53 2022] DELETE /identity/v3/projects/9abb1d2c44a24d6b81a51f048f2a4937 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:54.020972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:54.022736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:54.037720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450ef58-3694-4aa2-924f-b73d87f6cad8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:54.094760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d1a06-30f1-41c5-bf2a-debb220c44a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:54.095329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d1a06-30f1-41c5-bf2a-debb220c44a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:54.095608 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d1a06-30f1-41c5-bf2a-debb220c44a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:54.095880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d1a06-30f1-41c5-bf2a-debb220c44a9 admin admin] PATH_INFO: `/v3/users/746ec25646b5440ab513c380c89816cc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:54.097134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d1a06-30f1-41c5-bf2a-debb220c44a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=746ec25646b5440ab513c380c89816cc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:54.099055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d1a06-30f1-41c5-bf2a-debb220c44a9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:54.148645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6e9d1a06-30f1-41c5-bf2a-debb220c44a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746ec25646b5440ab513c380c89816cc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:54.155760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6e9d1a06-30f1-41c5-bf2a-debb220c44a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746ec25646b5440ab513c380c89816cc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:54.163584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:54.168414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6e9d1a06-30f1-41c5-bf2a-debb220c44a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746ec25646b5440ab513c380c89816cc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:54.169173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6e9d1a06-30f1-41c5-bf2a-debb220c44a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746ec25646b5440ab513c380c89816cc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:54.170263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6e9d1a06-30f1-41c5-bf2a-debb220c44a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746ec25646b5440ab513c380c89816cc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:54.183982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 978/1974] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:30:54 2022] DELETE /identity/v3/users/746ec25646b5440ab513c380c89816cc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:54.468308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:54.470322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:54.485503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3eea5-5016-48cf-8d54-5c7e82758724 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:54.543238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf023f-7e18-4a78-af8d-083e6f4d5816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:54.543876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf023f-7e18-4a78-af8d-083e6f4d5816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:54.544195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf023f-7e18-4a78-af8d-083e6f4d5816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:54.544496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf023f-7e18-4a78-af8d-083e6f4d5816 admin admin] PATH_INFO: `/v3/projects/b4357fec76ed44cd98ca24da64c6e5c5` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:54.545701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf023f-7e18-4a78-af8d-083e6f4d5816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4357fec76ed44cd98ca24da64c6e5c5)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:54.547620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf023f-7e18-4a78-af8d-083e6f4d5816 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:54.592884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a3cf023f-7e18-4a78-af8d-083e6f4d5816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4357fec76ed44cd98ca24da64c6e5c5', 'request_id': 'req-a3cf023f-7e18-4a78-af8d-083e6f4d5816'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:54.598650 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a3cf023f-7e18-4a78-af8d-083e6f4d5816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4357fec76ed44cd98ca24da64c6e5c5', 'request_id': 'req-a3cf023f-7e18-4a78-af8d-083e6f4d5816'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:54.605069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a3cf023f-7e18-4a78-af8d-083e6f4d5816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4357fec76ed44cd98ca24da64c6e5c5', 'request_id': 'req-a3cf023f-7e18-4a78-af8d-083e6f4d5816'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:54.618486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a3cf023f-7e18-4a78-af8d-083e6f4d5816 admin admin] The token cache is being invalidate because project b4357fec76ed44cd98ca24da64c6e5c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:30:54.618779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a3cf023f-7e18-4a78-af8d-083e6f4d5816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:54.619411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a3cf023f-7e18-4a78-af8d-083e6f4d5816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:30:54.621141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 997/1975] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:30:54 2022] DELETE /identity/v3/projects/b4357fec76ed44cd98ca24da64c6e5c5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:54.626945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adc6b3-5f7b-4529-ae9d-efb788922bf1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:54.627307 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adc6b3-5f7b-4529-ae9d-efb788922bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:54.627680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adc6b3-5f7b-4529-ae9d-efb788922bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:54.649378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e2adc6b3-5f7b-4529-ae9d-efb788922bf1 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:54.651179 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e2adc6b3-5f7b-4529-ae9d-efb788922bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:54.735270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 979/1976] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:54.743820 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d36ef4-33da-44a1-8f1e-dac96b18c272 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:54.744480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d36ef4-33da-44a1-8f1e-dac96b18c272 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:54.744954 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d36ef4-33da-44a1-8f1e-dac96b18c272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:54.783730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a4d36ef4-33da-44a1-8f1e-dac96b18c272 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:54.785953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a4d36ef4-33da-44a1-8f1e-dac96b18c272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:54.803365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 998/1977] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:54.804409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adc6b3-5f7b-4529-ae9d-efb788922bf1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:54.814971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adc6b3-5f7b-4529-ae9d-efb788922bf1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:54.820717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4ef02-023c-4883-88aa-634a8b27d7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:54.821334 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4ef02-023c-4883-88aa-634a8b27d7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:54.821619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4ef02-023c-4883-88aa-634a8b27d7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:54.821885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4ef02-023c-4883-88aa-634a8b27d7b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:54.822539 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4ef02-023c-4883-88aa-634a8b27d7b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:30:54.823290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4ef02-023c-4883-88aa-634a8b27d7b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:54.824722 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4ef02-023c-4883-88aa-634a8b27d7b4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:54.840769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 980/1978] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:30:54.845749 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d36ef4-33da-44a1-8f1e-dac96b18c272 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:54.856059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d36ef4-33da-44a1-8f1e-dac96b18c272 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:54.864482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674b6cb-2a63-47cf-8263-7bb755eaa625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:54.864841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674b6cb-2a63-47cf-8263-7bb755eaa625 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:54.865121 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674b6cb-2a63-47cf-8263-7bb755eaa625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:54.865398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674b6cb-2a63-47cf-8263-7bb755eaa625 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:54.866093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674b6cb-2a63-47cf-8263-7bb755eaa625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:54.867984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674b6cb-2a63-47cf-8263-7bb755eaa625 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:54.871727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4ef02-023c-4883-88aa-634a8b27d7b4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:54.874138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c5a4ef02-023c-4883-88aa-634a8b27d7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:54.952018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 999/1979] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:30:54 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:54.953061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674b6cb-2a63-47cf-8263-7bb755eaa625 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:54.966021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4ef02-023c-4883-88aa-634a8b27d7b4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:54.975803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:55.050715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674b6cb-2a63-47cf-8263-7bb755eaa625 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:55.052209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd2d10-9c83-4a10-9a31-0bb1edaae23a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:55.053316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd2d10-9c83-4a10-9a31-0bb1edaae23a service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:55.053596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd2d10-9c83-4a10-9a31-0bb1edaae23a service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:55.053863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd2d10-9c83-4a10-9a31-0bb1edaae23a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:55.057128 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-eddd2d10-9c83-4a10-9a31-0bb1edaae23a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:55.064421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14469652-12a9-4f76-ad82-c9609ebbacc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:55.065178 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14469652-12a9-4f76-ad82-c9609ebbacc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:55.065568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14469652-12a9-4f76-ad82-c9609ebbacc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:55.065945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14469652-12a9-4f76-ad82-c9609ebbacc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:55.066715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14469652-12a9-4f76-ad82-c9609ebbacc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:55.068868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14469652-12a9-4f76-ad82-c9609ebbacc6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:55.094436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1000/1980] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:30:54 2022] POST /identity/v3/users => generated 285 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:30:55.098331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14469652-12a9-4f76-ad82-c9609ebbacc6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:55.111643 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14469652-12a9-4f76-ad82-c9609ebbacc6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:55.118238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0fdd8-cd9a-4537-b8bd-1de5c64cddb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:55.118863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0fdd8-cd9a-4537-b8bd-1de5c64cddb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:55.119145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0fdd8-cd9a-4537-b8bd-1de5c64cddb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:55.119421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0fdd8-cd9a-4537-b8bd-1de5c64cddb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:55.120288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0fdd8-cd9a-4537-b8bd-1de5c64cddb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:55.122307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0fdd8-cd9a-4537-b8bd-1de5c64cddb4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:55.136388 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1001/1981] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:55.137067 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:30:55.140608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0fdd8-cd9a-4537-b8bd-1de5c64cddb4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:55.154778 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0fdd8-cd9a-4537-b8bd-1de5c64cddb4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:55.160296 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85231f-2faf-409e-b911-e91a14798309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:55.161001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85231f-2faf-409e-b911-e91a14798309 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:55.161260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85231f-2faf-409e-b911-e91a14798309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:55.161538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85231f-2faf-409e-b911-e91a14798309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:55.162193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85231f-2faf-409e-b911-e91a14798309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:55.163799 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85231f-2faf-409e-b911-e91a14798309 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:55.165755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd2d10-9c83-4a10-9a31-0bb1edaae23a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:55.167735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd2d10-9c83-4a10-9a31-0bb1edaae23a service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:55.176823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1002/1982] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:30:55.179561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 981/1983] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:30:55.180975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85231f-2faf-409e-b911-e91a14798309 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:55.191463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85231f-2faf-409e-b911-e91a14798309 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:55.196489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cddf7-26a3-42d6-8824-fe5fbc41d5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:55.197217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cddf7-26a3-42d6-8824-fe5fbc41d5c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:55.197412 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cddf7-26a3-42d6-8824-fe5fbc41d5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:55.197676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cddf7-26a3-42d6-8824-fe5fbc41d5c2 admin admin] PATH_INFO: `/v3/projects/d88908a970db4ee8a6ad8586538306f2/users/2e89e554b1854c1a902e6962d8103dca/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:55.218141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cddf7-26a3-42d6-8824-fe5fbc41d5c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d88908a970db4ee8a6ad8586538306f2, user_id=2e89e554b1854c1a902e6962d8103dca, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:55.220851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cddf7-26a3-42d6-8824-fe5fbc41d5c2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:55.235957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1003/1984] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:30:55 2022] PUT /identity/v3/projects/d88908a970db4ee8a6ad8586538306f2/users/2e89e554b1854c1a902e6962d8103dca/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:30:55.246877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d365816-b453-4854-b9e1-859a039390c8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:55.247168 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d365816-b453-4854-b9e1-859a039390c8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:55.247449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d365816-b453-4854-b9e1-859a039390c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:55.277081 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4d365816-b453-4854-b9e1-859a039390c8 None None] MFA Rules not processed for user `2e89e554b1854c1a902e6962d8103dca`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:30:55.278441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4d365816-b453-4854-b9e1-859a039390c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:55.345178 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 982/1985] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:30:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:30:55.357472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cddf7-26a3-42d6-8824-fe5fbc41d5c2 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:55.359280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d78cddf7-26a3-42d6-8824-fe5fbc41d5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:55.420997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cddf7-26a3-42d6-8824-fe5fbc41d5c2 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:55.424862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b250a-dc5a-4eb4-86ad-f4a31e6fa8d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0330c79c391d463b91f99e030c6dcc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:55.425474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b250a-dc5a-4eb4-86ad-f4a31e6fa8d6 service glance] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:55.425751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b250a-dc5a-4eb4-86ad-f4a31e6fa8d6 service glance] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:55.426020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b250a-dc5a-4eb4-86ad-f4a31e6fa8d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:55.434634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b250a-dc5a-4eb4-86ad-f4a31e6fa8d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:55.436324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b250a-dc5a-4eb4-86ad-f4a31e6fa8d6 service glance] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:55.446332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1004/1986] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:30:55.747009 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d365816-b453-4854-b9e1-859a039390c8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:30:55.819597 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d365816-b453-4854-b9e1-859a039390c8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:30:55.826161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce2241-8307-4580-8e9e-eb5e4a74bb66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:30:55.826792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce2241-8307-4580-8e9e-eb5e4a74bb66 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:30:55.827702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce2241-8307-4580-8e9e-eb5e4a74bb66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:30:55.827991 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce2241-8307-4580-8e9e-eb5e4a74bb66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:30:55.832804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2fce2241-8307-4580-8e9e-eb5e4a74bb66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:30:55.958287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce2241-8307-4580-8e9e-eb5e4a74bb66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:30:55.960196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce2241-8307-4580-8e9e-eb5e4a74bb66 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:30:55.977262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 983/1987] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:30:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:31:01.142591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b250a-dc5a-4eb4-86ad-f4a31e6fa8d6 service glance] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:01.144255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0b9b250a-dc5a-4eb4-86ad-f4a31e6fa8d6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:01.230430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b250a-dc5a-4eb4-86ad-f4a31e6fa8d6 service glance] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:01.236313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:01.236994 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:01.237298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:01.237565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] PATH_INFO: `/v3/users/cb46fd7dcdad4637a06a8af3653cf399` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:01.240523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb46fd7dcdad4637a06a8af3653cf399)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:01.242473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:01.250933 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:01.299828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb46fd7dcdad4637a06a8af3653cf399'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:01.300448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb46fd7dcdad4637a06a8af3653cf399'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:01.308081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb46fd7dcdad4637a06a8af3653cf399'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:01.330185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb46fd7dcdad4637a06a8af3653cf399'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:01.350022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb46fd7dcdad4637a06a8af3653cf399'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:01.360377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1005/1988] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:31:01 2022] DELETE /identity/v3/users/cb46fd7dcdad4637a06a8af3653cf399 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:01.362244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7df24-5cd8-4b16-aa1f-ddaa5d360bbb None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:01.362752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7df24-5cd8-4b16-aa1f-ddaa5d360bbb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:01.363166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7df24-5cd8-4b16-aa1f-ddaa5d360bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:01.385721 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-24c7df24-5cd8-4b16-aa1f-ddaa5d360bbb None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:01.387295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-24c7df24-5cd8-4b16-aa1f-ddaa5d360bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:01.407044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:01.468137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 984/1989] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:01.480092 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:01.482272 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:01.554450 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7b6e6-4fe0-47d8-bba3-fe94c3484a01 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:01.562835 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d13f33-7718-4621-aa32-54bb74230dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:01.563510 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d13f33-7718-4621-aa32-54bb74230dec service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:01.563790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d13f33-7718-4621-aa32-54bb74230dec service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:01.564303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d13f33-7718-4621-aa32-54bb74230dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:01.580484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d13f33-7718-4621-aa32-54bb74230dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:01.582082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d13f33-7718-4621-aa32-54bb74230dec service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:01.594012 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1006/1990] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:31:01.688872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7df24-5cd8-4b16-aa1f-ddaa5d360bbb None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:01.690402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-24c7df24-5cd8-4b16-aa1f-ddaa5d360bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:01.703273 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7df24-5cd8-4b16-aa1f-ddaa5d360bbb None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:01.708178 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:01.708976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:01.709258 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:01.709520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] PATH_INFO: `/v3/users/06a9bffefc614671a70315e6f994502f` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:01.710648 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06a9bffefc614671a70315e6f994502f)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:01.712668 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:01.772284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06a9bffefc614671a70315e6f994502f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:01.781007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06a9bffefc614671a70315e6f994502f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:01.798895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06a9bffefc614671a70315e6f994502f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:01.799652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06a9bffefc614671a70315e6f994502f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:01.803346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06a9bffefc614671a70315e6f994502f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:01.818301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 985/1991] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:31:01 2022] DELETE /identity/v3/users/06a9bffefc614671a70315e6f994502f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:01.822767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f6050-1203-4071-9d27-a76735c7f923 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:01.823104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f6050-1203-4071-9d27-a76735c7f923 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:01.823383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f6050-1203-4071-9d27-a76735c7f923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:01.843411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5a4f6050-1203-4071-9d27-a76735c7f923 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:01.844667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5a4f6050-1203-4071-9d27-a76735c7f923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:01.909535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:01.910982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:01.914201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1007/1992] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:01.925619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f6050-1203-4071-9d27-a76735c7f923 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:01.927001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5a4f6050-1203-4071-9d27-a76735c7f923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:01.931419 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c030623-015f-4206-a6bb-6820d58f7fa8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:01.943613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:01.944392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:01.944684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:01.944932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] PATH_INFO: `/v3/projects/86999cdbef3c41ff8b2a529f767108c4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:01.946354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86999cdbef3c41ff8b2a529f767108c4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:01.948310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:01.960949 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:01.996438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f6050-1203-4071-9d27-a76735c7f923 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:02.000796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86999cdbef3c41ff8b2a529f767108c4', 'request_id': 'req-5ea9a903-fd5d-4d60-befb-434b294f0302'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.009446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86999cdbef3c41ff8b2a529f767108c4', 'request_id': 'req-5ea9a903-fd5d-4d60-befb-434b294f0302'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.016798 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86999cdbef3c41ff8b2a529f767108c4', 'request_id': 'req-5ea9a903-fd5d-4d60-befb-434b294f0302'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.028401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] The token cache is being invalidate because project 86999cdbef3c41ff8b2a529f767108c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:31:02.028811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.029503 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.031045 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 986/1993] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:31:01 2022] DELETE /identity/v3/projects/86999cdbef3c41ff8b2a529f767108c4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:31:02.034986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:02.036304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:02.050388 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c791d1-f637-4588-8f7b-90ab55f173e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:02.051007 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c791d1-f637-4588-8f7b-90ab55f173e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:02.051287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c791d1-f637-4588-8f7b-90ab55f173e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:02.051560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c791d1-f637-4588-8f7b-90ab55f173e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:02.053300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-40c791d1-f637-4588-8f7b-90ab55f173e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:02.056702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:02.106026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9a903-fd5d-4d60-befb-434b294f0302 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:02.111284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:02.111919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:02.112244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:02.112521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] PATH_INFO: `/v3/users/f4edb28cb61b4b3484bf003d764e2bea` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:02.113650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4edb28cb61b4b3484bf003d764e2bea)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:02.115576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:02.120445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c791d1-f637-4588-8f7b-90ab55f173e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:02.121885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c791d1-f637-4588-8f7b-90ab55f173e6 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:02.131622 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1008/1994] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:31:02.174145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4edb28cb61b4b3484bf003d764e2bea'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.181738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4edb28cb61b4b3484bf003d764e2bea'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.199871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4edb28cb61b4b3484bf003d764e2bea'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.201449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4edb28cb61b4b3484bf003d764e2bea'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.202080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4edb28cb61b4b3484bf003d764e2bea'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.217838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 987/1995] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:31:02 2022] DELETE /identity/v3/users/f4edb28cb61b4b3484bf003d764e2bea => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:31:02.729386 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c791d1-f637-4588-8f7b-90ab55f173e6 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:02.731336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-40c791d1-f637-4588-8f7b-90ab55f173e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:02.757143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c791d1-f637-4588-8f7b-90ab55f173e6 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:02.832813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:02.833811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:02.878244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87b50f-3767-439c-86ce-a62ec203d71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:02.880064 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87b50f-3767-439c-86ce-a62ec203d71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:02.880403 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87b50f-3767-439c-86ce-a62ec203d71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:02.880682 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87b50f-3767-439c-86ce-a62ec203d71e admin admin] PATH_INFO: `/v3/projects/8c1ecca24a3f47579024d2e60c6c21d4` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:02.883035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87b50f-3767-439c-86ce-a62ec203d71e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c1ecca24a3f47579024d2e60c6c21d4)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:02.886981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87b50f-3767-439c-86ce-a62ec203d71e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:02.929868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:02.939806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-de87b50f-3767-439c-86ce-a62ec203d71e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c1ecca24a3f47579024d2e60c6c21d4', 'request_id': 'req-de87b50f-3767-439c-86ce-a62ec203d71e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.945049 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2166f-0cc8-4938-9d5f-71841ba40524 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:02.945676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-de87b50f-3767-439c-86ce-a62ec203d71e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c1ecca24a3f47579024d2e60c6c21d4', 'request_id': 'req-de87b50f-3767-439c-86ce-a62ec203d71e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.953493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-de87b50f-3767-439c-86ce-a62ec203d71e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c1ecca24a3f47579024d2e60c6c21d4', 'request_id': 'req-de87b50f-3767-439c-86ce-a62ec203d71e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.972917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-de87b50f-3767-439c-86ce-a62ec203d71e admin admin] The token cache is being invalidate because project 8c1ecca24a3f47579024d2e60c6c21d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:31:02.973736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-de87b50f-3767-439c-86ce-a62ec203d71e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.975404 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-de87b50f-3767-439c-86ce-a62ec203d71e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:02.977220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1009/1996] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:31:02 2022] DELETE /identity/v3/projects/8c1ecca24a3f47579024d2e60c6c21d4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:02.982178 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ea2bd-fa2b-4137-8f2d-7d4e06b019d9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:02.982587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ea2bd-fa2b-4137-8f2d-7d4e06b019d9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:02.982969 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ea2bd-fa2b-4137-8f2d-7d4e06b019d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:02.995341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:03.005084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a37ea2bd-fa2b-4137-8f2d-7d4e06b019d9 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:03.006333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a37ea2bd-fa2b-4137-8f2d-7d4e06b019d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:03.014330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:03.014966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:03.015245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:03.015517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] PATH_INFO: `/v3/projects/3eb6148713864fa1a6ab8e45430927fe` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:03.016673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eb6148713864fa1a6ab8e45430927fe)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:03.018584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:03.068510 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1010/1997] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:31:03.076603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dabb9f-6183-47f5-b474-a7b17a7a2b5d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:03.076888 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dabb9f-6183-47f5-b474-a7b17a7a2b5d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:03.077112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eb6148713864fa1a6ab8e45430927fe', 'request_id': 'req-604fa52f-c63f-47c8-a83f-6b275c373da4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.077232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dabb9f-6183-47f5-b474-a7b17a7a2b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:03.084277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eb6148713864fa1a6ab8e45430927fe', 'request_id': 'req-604fa52f-c63f-47c8-a83f-6b275c373da4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.091853 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eb6148713864fa1a6ab8e45430927fe', 'request_id': 'req-604fa52f-c63f-47c8-a83f-6b275c373da4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.097661 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-75dabb9f-6183-47f5-b474-a7b17a7a2b5d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:03.098875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-75dabb9f-6183-47f5-b474-a7b17a7a2b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:03.102552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] The token cache is being invalidate because project 3eb6148713864fa1a6ab8e45430927fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:31:03.102843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.103648 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.105155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 988/1998] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:31:02 2022] DELETE /identity/v3/projects/3eb6148713864fa1a6ab8e45430927fe => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:31:03.108937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:03.110219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:03.159987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1011/1999] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:31:03.164486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dabb9f-6183-47f5-b474-a7b17a7a2b5d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:03.173509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dabb9f-6183-47f5-b474-a7b17a7a2b5d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:03.175852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fa52f-c63f-47c8-a83f-6b275c373da4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:03.178244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92dcc9-700a-43cd-bd0b-0877716fe08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:03.178844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92dcc9-700a-43cd-bd0b-0877716fe08c admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:03.179125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92dcc9-700a-43cd-bd0b-0877716fe08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:03.179401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92dcc9-700a-43cd-bd0b-0877716fe08c admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:03.180084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92dcc9-700a-43cd-bd0b-0877716fe08c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:31:03.180396 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92dcc9-700a-43cd-bd0b-0877716fe08c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:03.180968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:03.181594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:03.181875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:03.181985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92dcc9-700a-43cd-bd0b-0877716fe08c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:03.182147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] PATH_INFO: `/v3/users/ceb104f09a174a4f8fa8d4ebacb4037b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:03.183307 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceb104f09a174a4f8fa8d4ebacb4037b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:03.186002 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:03.194307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1012/2000] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:31:03.198581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92dcc9-700a-43cd-bd0b-0877716fe08c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:03.209096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92dcc9-700a-43cd-bd0b-0877716fe08c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:03.214240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1aebd-c6a5-4735-ab51-3109db4b5220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:03.214840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1aebd-c6a5-4735-ab51-3109db4b5220 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:03.215120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1aebd-c6a5-4735-ab51-3109db4b5220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:03.215412 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1aebd-c6a5-4735-ab51-3109db4b5220 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:03.216411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1aebd-c6a5-4735-ab51-3109db4b5220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:03.218318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1aebd-c6a5-4735-ab51-3109db4b5220 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:03.235932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceb104f09a174a4f8fa8d4ebacb4037b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.243108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceb104f09a174a4f8fa8d4ebacb4037b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.246117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1013/2001] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:31:03 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:31:03.251086 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1aebd-c6a5-4735-ab51-3109db4b5220 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:03.256643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceb104f09a174a4f8fa8d4ebacb4037b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.257744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceb104f09a174a4f8fa8d4ebacb4037b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.258667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceb104f09a174a4f8fa8d4ebacb4037b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.261299 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:03.279302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 989/2002] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:31:03 2022] DELETE /identity/v3/users/ceb104f09a174a4f8fa8d4ebacb4037b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:31:03.290370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:03.327983 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1aebd-c6a5-4735-ab51-3109db4b5220 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:03.427767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2d555-216b-4df6-84bf-fda4739de358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:03.428535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2d555-216b-4df6-84bf-fda4739de358 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:03.428817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2d555-216b-4df6-84bf-fda4739de358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:03.429090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2d555-216b-4df6-84bf-fda4739de358 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:03.429725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2d555-216b-4df6-84bf-fda4739de358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:03.431691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2d555-216b-4df6-84bf-fda4739de358 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:03.460428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1014/2003] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:31:03 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:31:03.462023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:03.463461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:03.477281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73b013-212c-40e1-907f-2828a2f90971 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:03.482336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4cd22-4ba6-481d-b22d-6072ef0ee239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:03.482876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4cd22-4ba6-481d-b22d-6072ef0ee239 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:03.483151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4cd22-4ba6-481d-b22d-6072ef0ee239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:03.483420 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4cd22-4ba6-481d-b22d-6072ef0ee239 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:03.484119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4cd22-4ba6-481d-b22d-6072ef0ee239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:03.485722 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4cd22-4ba6-481d-b22d-6072ef0ee239 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:03.498935 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 990/2004] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:31:03.501155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2d555-216b-4df6-84bf-fda4739de358 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:03.515541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2d555-216b-4df6-84bf-fda4739de358 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:03.520422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383fa3c-2536-49c6-9fdf-024e7c13de3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:03.521054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383fa3c-2536-49c6-9fdf-024e7c13de3f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:03.521349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383fa3c-2536-49c6-9fdf-024e7c13de3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:03.521694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383fa3c-2536-49c6-9fdf-024e7c13de3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:03.522355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383fa3c-2536-49c6-9fdf-024e7c13de3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:03.524624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383fa3c-2536-49c6-9fdf-024e7c13de3f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:03.549885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1015/2005] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:03.552989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4cd22-4ba6-481d-b22d-6072ef0ee239 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:03.563871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4cd22-4ba6-481d-b22d-6072ef0ee239 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:03.568491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce848e-4620-41cc-bb31-cf61b1333982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:03.569150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce848e-4620-41cc-bb31-cf61b1333982 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:03.569431 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce848e-4620-41cc-bb31-cf61b1333982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:03.569705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce848e-4620-41cc-bb31-cf61b1333982 admin admin] PATH_INFO: `/v3/projects/16e859bc783a445f9b951dd2363fa0ce/users/8424e74e566d4bf3b8840b5d5c20aa1f/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:03.589748 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce848e-4620-41cc-bb31-cf61b1333982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e859bc783a445f9b951dd2363fa0ce, user_id=8424e74e566d4bf3b8840b5d5c20aa1f, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:03.593543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce848e-4620-41cc-bb31-cf61b1333982 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:03.616041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 991/2006] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:31:03 2022] PUT /identity/v3/projects/16e859bc783a445f9b951dd2363fa0ce/users/8424e74e566d4bf3b8840b5d5c20aa1f/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:03.618395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1401a-40b8-4fa2-b9b1-d44ce8e8274c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:03.619410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1401a-40b8-4fa2-b9b1-d44ce8e8274c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:03.619693 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1401a-40b8-4fa2-b9b1-d44ce8e8274c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:03.641661 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-4db1401a-40b8-4fa2-b9b1-d44ce8e8274c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:03.643644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4db1401a-40b8-4fa2-b9b1-d44ce8e8274c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:03.688374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce848e-4620-41cc-bb31-cf61b1333982 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:03.690644 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-95ce848e-4620-41cc-bb31-cf61b1333982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:03.724561 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1016/2007] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:31:03.736639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1401a-40b8-4fa2-b9b1-d44ce8e8274c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:03.736639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4db1401a-40b8-4fa2-b9b1-d44ce8e8274c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:03.822112 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1401a-40b8-4fa2-b9b1-d44ce8e8274c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:03.826857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f347af2-64f2-45be-bb63-ead1cdbba256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:03.827475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f347af2-64f2-45be-bb63-ead1cdbba256 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:03.827756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f347af2-64f2-45be-bb63-ead1cdbba256 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:03.828238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f347af2-64f2-45be-bb63-ead1cdbba256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:03.833010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce848e-4620-41cc-bb31-cf61b1333982 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:03.839041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa64e5-2416-419a-a9e6-abc615155e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:03.839375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f347af2-64f2-45be-bb63-ead1cdbba256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:03.840070 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa64e5-2416-419a-a9e6-abc615155e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:03.840542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa64e5-2416-419a-a9e6-abc615155e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:03.840841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f347af2-64f2-45be-bb63-ead1cdbba256 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:03.841247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa64e5-2416-419a-a9e6-abc615155e78 admin admin] PATH_INFO: `/v3/projects/1371be62105343e8a845703cfdb30980` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:03.842907 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa64e5-2416-419a-a9e6-abc615155e78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1371be62105343e8a845703cfdb30980)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:03.845087 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa64e5-2416-419a-a9e6-abc615155e78 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:03.847202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:03.853267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1017/2008] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:31:03.917994 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a6aa64e5-2416-419a-a9e6-abc615155e78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1371be62105343e8a845703cfdb30980', 'request_id': 'req-a6aa64e5-2416-419a-a9e6-abc615155e78'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.931564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a6aa64e5-2416-419a-a9e6-abc615155e78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1371be62105343e8a845703cfdb30980', 'request_id': 'req-a6aa64e5-2416-419a-a9e6-abc615155e78'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.942729 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a6aa64e5-2416-419a-a9e6-abc615155e78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1371be62105343e8a845703cfdb30980', 'request_id': 'req-a6aa64e5-2416-419a-a9e6-abc615155e78'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.960204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a6aa64e5-2416-419a-a9e6-abc615155e78 admin admin] The token cache is being invalidate because project 1371be62105343e8a845703cfdb30980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:31:03.960552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a6aa64e5-2416-419a-a9e6-abc615155e78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.961174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a6aa64e5-2416-419a-a9e6-abc615155e78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:03.963352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 992/2009] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:31:03 2022] DELETE /identity/v3/projects/1371be62105343e8a845703cfdb30980 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:31:03.970065 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5223ac-3ba3-44c1-ba77-241baff2e12a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:03.970347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5223ac-3ba3-44c1-ba77-241baff2e12a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:03.970616 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5223ac-3ba3-44c1-ba77-241baff2e12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:03.993863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ef5223ac-3ba3-44c1-ba77-241baff2e12a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:03.995217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ef5223ac-3ba3-44c1-ba77-241baff2e12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:04.078653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1018/2010] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:04.087126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a300a56-485f-4e92-b7c3-bfeb8e459905 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:04.087599 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a300a56-485f-4e92-b7c3-bfeb8e459905 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:04.087946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a300a56-485f-4e92-b7c3-bfeb8e459905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:04.110627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5a300a56-485f-4e92-b7c3-bfeb8e459905 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:04.111853 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5a300a56-485f-4e92-b7c3-bfeb8e459905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:04.126713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 993/2011] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:04.136663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5223ac-3ba3-44c1-ba77-241baff2e12a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:04.151010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5223ac-3ba3-44c1-ba77-241baff2e12a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:04.157330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44eb110-c9c6-4da8-8074-371f1f2babcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:04.158356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44eb110-c9c6-4da8-8074-371f1f2babcb admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:04.158784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44eb110-c9c6-4da8-8074-371f1f2babcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:04.159747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44eb110-c9c6-4da8-8074-371f1f2babcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:04.160673 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44eb110-c9c6-4da8-8074-371f1f2babcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:31:04.161145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44eb110-c9c6-4da8-8074-371f1f2babcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:04.162978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44eb110-c9c6-4da8-8074-371f1f2babcb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:04.181231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1019/2012] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:31:04.186800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a300a56-485f-4e92-b7c3-bfeb8e459905 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:04.197568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a300a56-485f-4e92-b7c3-bfeb8e459905 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:04.202718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8a25d-9fc3-414d-8a7a-8cd3f7ed0bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:04.203346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8a25d-9fc3-414d-8a7a-8cd3f7ed0bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:04.203629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8a25d-9fc3-414d-8a7a-8cd3f7ed0bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:04.203908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8a25d-9fc3-414d-8a7a-8cd3f7ed0bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:04.204610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8a25d-9fc3-414d-8a7a-8cd3f7ed0bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:04.206536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8a25d-9fc3-414d-8a7a-8cd3f7ed0bd5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:04.240513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 994/2013] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:31:04 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:31:04.242118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44eb110-c9c6-4da8-8074-371f1f2babcb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:04.254251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44eb110-c9c6-4da8-8074-371f1f2babcb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:04.318410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f1cbb-e3ba-47d4-bb73-1da449ac53b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:04.319044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f1cbb-e3ba-47d4-bb73-1da449ac53b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:04.319327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f1cbb-e3ba-47d4-bb73-1da449ac53b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:04.319600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f1cbb-e3ba-47d4-bb73-1da449ac53b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:04.320284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f1cbb-e3ba-47d4-bb73-1da449ac53b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:04.322246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f1cbb-e3ba-47d4-bb73-1da449ac53b1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:04.348895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1020/2014] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:31:04 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:31:04.350147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8a25d-9fc3-414d-8a7a-8cd3f7ed0bd5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:04.369143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8a25d-9fc3-414d-8a7a-8cd3f7ed0bd5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:04.375671 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488a7fa-1957-4bf4-9439-3fc7832a7b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:04.376487 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488a7fa-1957-4bf4-9439-3fc7832a7b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:04.377019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488a7fa-1957-4bf4-9439-3fc7832a7b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:04.377373 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488a7fa-1957-4bf4-9439-3fc7832a7b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:04.378139 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488a7fa-1957-4bf4-9439-3fc7832a7b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:04.379976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488a7fa-1957-4bf4-9439-3fc7832a7b3f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:04.396656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 995/2015] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:04.398970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f1cbb-e3ba-47d4-bb73-1da449ac53b1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:04.413528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f1cbb-e3ba-47d4-bb73-1da449ac53b1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:04.423975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52e5d3-2a90-4e64-a3c6-2431a4239ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:04.423975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52e5d3-2a90-4e64-a3c6-2431a4239ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:04.423975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52e5d3-2a90-4e64-a3c6-2431a4239ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:04.423975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52e5d3-2a90-4e64-a3c6-2431a4239ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:04.424382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52e5d3-2a90-4e64-a3c6-2431a4239ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:04.425995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52e5d3-2a90-4e64-a3c6-2431a4239ec3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:04.443146 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1021/2016] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:04.450976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488a7fa-1957-4bf4-9439-3fc7832a7b3f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:04.482964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488a7fa-1957-4bf4-9439-3fc7832a7b3f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:04.489533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780e66d-e68b-4b08-a824-6bd3fc627b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:04.490360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780e66d-e68b-4b08-a824-6bd3fc627b4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:04.490626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780e66d-e68b-4b08-a824-6bd3fc627b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:04.490902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780e66d-e68b-4b08-a824-6bd3fc627b4f admin admin] PATH_INFO: `/v3/projects/2a9ef16092824511b11494d4388e3c58/users/b67b9e5012f34ab4a17ab579131a986e/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:04.494443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:04.518435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780e66d-e68b-4b08-a824-6bd3fc627b4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a9ef16092824511b11494d4388e3c58, user_id=b67b9e5012f34ab4a17ab579131a986e, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:04.521503 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780e66d-e68b-4b08-a824-6bd3fc627b4f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:04.552207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 996/2017] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:31:04 2022] PUT /identity/v3/projects/2a9ef16092824511b11494d4388e3c58/users/b67b9e5012f34ab4a17ab579131a986e/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:04.556863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32d2bb-10c8-4b0f-a477-e7a993a0967c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:04.557160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32d2bb-10c8-4b0f-a477-e7a993a0967c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:04.557424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32d2bb-10c8-4b0f-a477-e7a993a0967c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:04.591656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8d32d2bb-10c8-4b0f-a477-e7a993a0967c None None] MFA Rules not processed for user `b67b9e5012f34ab4a17ab579131a986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:04.592956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8d32d2bb-10c8-4b0f-a477-e7a993a0967c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:04.655450 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:04.667101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1022/2018] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:04.678754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780e66d-e68b-4b08-a824-6bd3fc627b4f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:04.680589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b780e66d-e68b-4b08-a824-6bd3fc627b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:04.746991 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780e66d-e68b-4b08-a824-6bd3fc627b4f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:04.752342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b059601-88ff-4718-93b5-22f5f7eddab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:04.752964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b059601-88ff-4718-93b5-22f5f7eddab1 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:04.753369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b059601-88ff-4718-93b5-22f5f7eddab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:04.753658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b059601-88ff-4718-93b5-22f5f7eddab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:04.763498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b059601-88ff-4718-93b5-22f5f7eddab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:04.764983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b059601-88ff-4718-93b5-22f5f7eddab1 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:04.785289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 997/2019] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:31:05.699628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240c139-dcee-4c42-9ed9-3146037b04f7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:05.700787 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240c139-dcee-4c42-9ed9-3146037b04f7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:05.700787 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240c139-dcee-4c42-9ed9-3146037b04f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:05.730830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2240c139-dcee-4c42-9ed9-3146037b04f7 None None] MFA Rules not processed for user `8424e74e566d4bf3b8840b5d5c20aa1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:05.732408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2240c139-dcee-4c42-9ed9-3146037b04f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:05.813352 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1023/2020] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:05.827014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b059601-88ff-4718-93b5-22f5f7eddab1 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:05.828851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0b059601-88ff-4718-93b5-22f5f7eddab1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:05.840589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b059601-88ff-4718-93b5-22f5f7eddab1 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:05.845196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cdd22f-eeaa-4768-b69c-2ccf42512a3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:05.845870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdd22f-eeaa-4768-b69c-2ccf42512a3c service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:05.846215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdd22f-eeaa-4768-b69c-2ccf42512a3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:05.846555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdd22f-eeaa-4768-b69c-2ccf42512a3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:05.855687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cdd22f-eeaa-4768-b69c-2ccf42512a3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:05.857323 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cdd22f-eeaa-4768-b69c-2ccf42512a3c service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:05.868052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 998/2021] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:31:06.240934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e4743-5f57-42dc-b3b7-206b6a8642bd None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:06.241273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e4743-5f57-42dc-b3b7-206b6a8642bd None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:06.241498 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e4743-5f57-42dc-b3b7-206b6a8642bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:06.284830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-515e4743-5f57-42dc-b3b7-206b6a8642bd None None] MFA Rules not processed for user `b67b9e5012f34ab4a17ab579131a986e`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:06.286050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-515e4743-5f57-42dc-b3b7-206b6a8642bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:06.295450 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1024/2022] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:08.340674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cbb84-f39b-4261-8a47-1690fed35434 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:08.341433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cbb84-f39b-4261-8a47-1690fed35434 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:08.341433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9cbb84-f39b-4261-8a47-1690fed35434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:08.361252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-bd9cbb84-f39b-4261-8a47-1690fed35434 None None] MFA Rules not processed for user `8424e74e566d4bf3b8840b5d5c20aa1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:08.362465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bd9cbb84-f39b-4261-8a47-1690fed35434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:08.369268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 999/2023] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:16.278022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=37 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:18.355505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:40.970515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd477946-ce0b-4899-8fbb-4e755d84e0ab None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:40.971880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd477946-ce0b-4899-8fbb-4e755d84e0ab None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:40.971880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd477946-ce0b-4899-8fbb-4e755d84e0ab None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:40.973145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1025/2024] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 05:31:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 05:31:40.991815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc5900-00fe-4a0c-8483-290ee3a99232 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:40.992344 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc5900-00fe-4a0c-8483-290ee3a99232 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:40.992451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc5900-00fe-4a0c-8483-290ee3a99232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:41.015730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a5fc5900-00fe-4a0c-8483-290ee3a99232 None None] MFA Rules not processed for user `4e985df8ad6c4bc7b3cd1328924e3ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:41.016970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a5fc5900-00fe-4a0c-8483-290ee3a99232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:41.090446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd477946-ce0b-4899-8fbb-4e755d84e0ab None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:41.092341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cd477946-ce0b-4899-8fbb-4e755d84e0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:41.114651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1000/2025] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 05:31:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:41.128321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc5900-00fe-4a0c-8483-290ee3a99232 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:41.130081 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a5fc5900-00fe-4a0c-8483-290ee3a99232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:41.146248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc5900-00fe-4a0c-8483-290ee3a99232 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:41.151639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d255d0-7f23-482f-bbf0-26643d0498a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:41.152390 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d255d0-7f23-482f-bbf0-26643d0498a3 service placement] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:41.152658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d255d0-7f23-482f-bbf0-26643d0498a3 service placement] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:41.152929 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d255d0-7f23-482f-bbf0-26643d0498a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:41.162778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d255d0-7f23-482f-bbf0-26643d0498a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:41.164606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d255d0-7f23-482f-bbf0-26643d0498a3 service placement] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:41.172777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd477946-ce0b-4899-8fbb-4e755d84e0ab None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:41.174865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1001/2026] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:31:41.177532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db863a46-6709-4770-a5be-1f1fb5517387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:41.178508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db863a46-6709-4770-a5be-1f1fb5517387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:41.178831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db863a46-6709-4770-a5be-1f1fb5517387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:41.179103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db863a46-6709-4770-a5be-1f1fb5517387 admin admin] PATH_INFO: `/v3/users/065c988c57fa47e9b58af2914ff6306d` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:41.180605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db863a46-6709-4770-a5be-1f1fb5517387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=065c988c57fa47e9b58af2914ff6306d)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:41.182246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db863a46-6709-4770-a5be-1f1fb5517387 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:41.239552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-db863a46-6709-4770-a5be-1f1fb5517387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '065c988c57fa47e9b58af2914ff6306d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:41.240266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-db863a46-6709-4770-a5be-1f1fb5517387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '065c988c57fa47e9b58af2914ff6306d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:41.251238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-db863a46-6709-4770-a5be-1f1fb5517387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '065c988c57fa47e9b58af2914ff6306d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:41.261008 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-db863a46-6709-4770-a5be-1f1fb5517387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '065c988c57fa47e9b58af2914ff6306d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:41.276074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-db863a46-6709-4770-a5be-1f1fb5517387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '065c988c57fa47e9b58af2914ff6306d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:41.282864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1026/2027] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:31:41 2022] DELETE /identity/v3/users/065c988c57fa47e9b58af2914ff6306d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:31:41.675259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d255d0-7f23-482f-bbf0-26643d0498a3 service placement] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:41.688962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d255d0-7f23-482f-bbf0-26643d0498a3 service placement] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:41.816938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af915605-c699-428c-a554-c4fd454714bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:41.817807 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af915605-c699-428c-a554-c4fd454714bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:41.819318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af915605-c699-428c-a554-c4fd454714bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:41.819318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af915605-c699-428c-a554-c4fd454714bf admin admin] PATH_INFO: `/v3/projects/431b72dd50e44a53a597b28404ace63c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:41.820067 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af915605-c699-428c-a554-c4fd454714bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=431b72dd50e44a53a597b28404ace63c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:41.822210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af915605-c699-428c-a554-c4fd454714bf admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:41.896183 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-af915605-c699-428c-a554-c4fd454714bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '431b72dd50e44a53a597b28404ace63c', 'request_id': 'req-af915605-c699-428c-a554-c4fd454714bf'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:41.905069 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-af915605-c699-428c-a554-c4fd454714bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '431b72dd50e44a53a597b28404ace63c', 'request_id': 'req-af915605-c699-428c-a554-c4fd454714bf'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:41.912086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-af915605-c699-428c-a554-c4fd454714bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '431b72dd50e44a53a597b28404ace63c', 'request_id': 'req-af915605-c699-428c-a554-c4fd454714bf'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:41.924759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-af915605-c699-428c-a554-c4fd454714bf admin admin] The token cache is being invalidate because project 431b72dd50e44a53a597b28404ace63c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:31:41.925051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-af915605-c699-428c-a554-c4fd454714bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:41.925784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-af915605-c699-428c-a554-c4fd454714bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:41.927336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1002/2028] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:31:41 2022] DELETE /identity/v3/projects/431b72dd50e44a53a597b28404ace63c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:41.932505 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191106-e5c0-4a1e-8ca3-78ec50fc3a4c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:41.932792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191106-e5c0-4a1e-8ca3-78ec50fc3a4c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:41.933060 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191106-e5c0-4a1e-8ca3-78ec50fc3a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:41.953074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-94191106-e5c0-4a1e-8ca3-78ec50fc3a4c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:41.954275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-94191106-e5c0-4a1e-8ca3-78ec50fc3a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:42.018105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1027/2029] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:42.026015 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac89808-95fc-492a-b32c-00594c8242b6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:42.026466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac89808-95fc-492a-b32c-00594c8242b6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:42.026579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac89808-95fc-492a-b32c-00594c8242b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:42.046789 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5ac89808-95fc-492a-b32c-00594c8242b6 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:42.048022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5ac89808-95fc-492a-b32c-00594c8242b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:42.056661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1003/2030] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:42.060961 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94191106-e5c0-4a1e-8ca3-78ec50fc3a4c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:42.071271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94191106-e5c0-4a1e-8ca3-78ec50fc3a4c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:42.075880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea30be0-69f1-4d67-a5d9-a12388681c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:42.076508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea30be0-69f1-4d67-a5d9-a12388681c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:42.076788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea30be0-69f1-4d67-a5d9-a12388681c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:42.077063 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea30be0-69f1-4d67-a5d9-a12388681c58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:42.077741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea30be0-69f1-4d67-a5d9-a12388681c58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:31:42.078045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea30be0-69f1-4d67-a5d9-a12388681c58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:42.079640 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea30be0-69f1-4d67-a5d9-a12388681c58 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:42.091832 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1028/2031] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:31:42.096184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac89808-95fc-492a-b32c-00594c8242b6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:42.105647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac89808-95fc-492a-b32c-00594c8242b6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:42.110364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf197f-df1d-4dd2-9b52-dea8ddc2e56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:42.110947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf197f-df1d-4dd2-9b52-dea8ddc2e56a admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:42.111450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf197f-df1d-4dd2-9b52-dea8ddc2e56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:42.111797 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf197f-df1d-4dd2-9b52-dea8ddc2e56a admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:42.112486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bf197f-df1d-4dd2-9b52-dea8ddc2e56a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:42.114403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bf197f-df1d-4dd2-9b52-dea8ddc2e56a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:42.118829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:42.144382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1004/2032] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:31:42 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:31:42.148618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea30be0-69f1-4d67-a5d9-a12388681c58 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:42.158601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea30be0-69f1-4d67-a5d9-a12388681c58 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:42.162131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:42.216302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23e82b-ab2c-4020-a923-20b8c1d5c4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:42.216901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23e82b-ab2c-4020-a923-20b8c1d5c4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:42.217177 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23e82b-ab2c-4020-a923-20b8c1d5c4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:42.217443 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23e82b-ab2c-4020-a923-20b8c1d5c4b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:42.218072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23e82b-ab2c-4020-a923-20b8c1d5c4b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:42.219972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23e82b-ab2c-4020-a923-20b8c1d5c4b5 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:42.241810 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1029/2033] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:31:42 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:31:42.245963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf197f-df1d-4dd2-9b52-dea8ddc2e56a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:42.256605 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf197f-df1d-4dd2-9b52-dea8ddc2e56a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:42.261502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc8acb-cd4d-47e3-9bfe-7c98bfd3ff2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:42.262077 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc8acb-cd4d-47e3-9bfe-7c98bfd3ff2d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:42.262356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc8acb-cd4d-47e3-9bfe-7c98bfd3ff2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:42.262626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc8acb-cd4d-47e3-9bfe-7c98bfd3ff2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:42.263262 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc8acb-cd4d-47e3-9bfe-7c98bfd3ff2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:42.264880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc8acb-cd4d-47e3-9bfe-7c98bfd3ff2d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:42.274888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1005/2034] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:42.278852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23e82b-ab2c-4020-a923-20b8c1d5c4b5 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:42.289289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23e82b-ab2c-4020-a923-20b8c1d5c4b5 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:42.293878 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d4c00-144c-4115-ab93-bba3f0337537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:42.294466 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d4c00-144c-4115-ab93-bba3f0337537 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:42.294742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d4c00-144c-4115-ab93-bba3f0337537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:42.295014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d4c00-144c-4115-ab93-bba3f0337537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:42.295645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d4c00-144c-4115-ab93-bba3f0337537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:42.297267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d4c00-144c-4115-ab93-bba3f0337537 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:42.308837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1030/2035] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:42.312885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc8acb-cd4d-47e3-9bfe-7c98bfd3ff2d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:42.322867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc8acb-cd4d-47e3-9bfe-7c98bfd3ff2d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:42.327405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2e05c-9049-410c-9e81-18cf70327b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:42.328202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2e05c-9049-410c-9e81-18cf70327b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:42.328486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2e05c-9049-410c-9e81-18cf70327b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:42.328762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2e05c-9049-410c-9e81-18cf70327b32 admin admin] PATH_INFO: `/v3/projects/d0d73e5ef724423dac1a4b235a4d8b94/users/e12236a4b4214c8593e619a97b62a129/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:42.343513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd2e05c-9049-410c-9e81-18cf70327b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0d73e5ef724423dac1a4b235a4d8b94, user_id=e12236a4b4214c8593e619a97b62a129, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:42.346727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd2e05c-9049-410c-9e81-18cf70327b32 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:42.358877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1006/2036] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:31:42 2022] PUT /identity/v3/projects/d0d73e5ef724423dac1a4b235a4d8b94/users/e12236a4b4214c8593e619a97b62a129/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:42.370186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233951c4-052c-4f10-a464-e550e1d2b074 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:42.370473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233951c4-052c-4f10-a464-e550e1d2b074 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:42.370751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233951c4-052c-4f10-a464-e550e1d2b074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:42.398870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-233951c4-052c-4f10-a464-e550e1d2b074 None None] MFA Rules not processed for user `e12236a4b4214c8593e619a97b62a129`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:42.400113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-233951c4-052c-4f10-a464-e550e1d2b074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:42.466945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1031/2037] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:42.479121 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2e05c-9049-410c-9e81-18cf70327b32 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:42.480895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0dd2e05c-9049-410c-9e81-18cf70327b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:42.556406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2e05c-9049-410c-9e81-18cf70327b32 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:42.561966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc1636-0f81-4d92-b9e3-44edb647b535 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:42.562572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1636-0f81-4d92-b9e3-44edb647b535 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:42.562855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1636-0f81-4d92-b9e3-44edb647b535 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:42.563130 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1636-0f81-4d92-b9e3-44edb647b535 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:42.571520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc1636-0f81-4d92-b9e3-44edb647b535 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:42.574021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc1636-0f81-4d92-b9e3-44edb647b535 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:42.583539 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1007/2038] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:31:42.819173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233951c4-052c-4f10-a464-e550e1d2b074 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:42.907691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233951c4-052c-4f10-a464-e550e1d2b074 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:42.913649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:42.914355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:42.914715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:42.915054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] PATH_INFO: `/v3/users/e12236a4b4214c8593e619a97b62a129` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:42.916322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e12236a4b4214c8593e619a97b62a129)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:42.918243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:42.935086 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:42.968069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e12236a4b4214c8593e619a97b62a129'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:42.968731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e12236a4b4214c8593e619a97b62a129'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:42.975765 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e12236a4b4214c8593e619a97b62a129'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:42.982991 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e12236a4b4214c8593e619a97b62a129'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:42.994915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e12236a4b4214c8593e619a97b62a129'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:43.000663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1032/2039] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:31:42 2022] DELETE /identity/v3/users/e12236a4b4214c8593e619a97b62a129 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:43.005634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5821f-90c0-43ea-bd8e-09e3fd11f8db None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:43.005922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5821f-90c0-43ea-bd8e-09e3fd11f8db None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:43.006195 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5821f-90c0-43ea-bd8e-09e3fd11f8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:43.026202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-72a5821f-90c0-43ea-bd8e-09e3fd11f8db None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:43.027443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-72a5821f-90c0-43ea-bd8e-09e3fd11f8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:43.086627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1008/2040] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:43.096210 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:43.097752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:43.157954 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b65c7c-050b-40f5-a5aa-232b2665c0fc admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:43.161623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df34f6-1410-40c5-9b40-977e96c7864a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:43.162233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df34f6-1410-40c5-9b40-977e96c7864a service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:43.162513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df34f6-1410-40c5-9b40-977e96c7864a service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:43.162769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df34f6-1410-40c5-9b40-977e96c7864a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:43.172250 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df34f6-1410-40c5-9b40-977e96c7864a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:43.174197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df34f6-1410-40c5-9b40-977e96c7864a service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:43.183972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1033/2041] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:31:43.451411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5821f-90c0-43ea-bd8e-09e3fd11f8db None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:43.452861 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-72a5821f-90c0-43ea-bd8e-09e3fd11f8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:43.465792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5821f-90c0-43ea-bd8e-09e3fd11f8db None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:43.470408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:43.471063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:43.471428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:43.471611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a admin admin] PATH_INFO: `/v3/projects/d0d73e5ef724423dac1a4b235a4d8b94` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:43.472785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0d73e5ef724423dac1a4b235a4d8b94)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:43.474731 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:43.509386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:43.524606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0d73e5ef724423dac1a4b235a4d8b94', 'request_id': 'req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:43.531621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0d73e5ef724423dac1a4b235a4d8b94', 'request_id': 'req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:43.537941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0d73e5ef724423dac1a4b235a4d8b94', 'request_id': 'req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:43.548360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a admin admin] The token cache is being invalidate because project d0d73e5ef724423dac1a4b235a4d8b94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:31:43.548692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:43.549426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e3bd9722-0017-4485-9d1f-0e6c2dbf468a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:43.550983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1009/2042] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:31:43 2022] DELETE /identity/v3/projects/d0d73e5ef724423dac1a4b235a4d8b94 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:31:43.555918 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b664b-b0ec-4ac9-aed6-68fd8ff54593 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:43.556235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b664b-b0ec-4ac9-aed6-68fd8ff54593 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:43.556508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b664b-b0ec-4ac9-aed6-68fd8ff54593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:43.576217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-032b664b-b0ec-4ac9-aed6-68fd8ff54593 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:43.577415 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-032b664b-b0ec-4ac9-aed6-68fd8ff54593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:43.635922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1034/2043] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:43.644783 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be6945-1ec6-4841-a2b3-9f6808df9569 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:43.645076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be6945-1ec6-4841-a2b3-9f6808df9569 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:43.645358 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be6945-1ec6-4841-a2b3-9f6808df9569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:43.665263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-84be6945-1ec6-4841-a2b3-9f6808df9569 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:43.666485 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-84be6945-1ec6-4841-a2b3-9f6808df9569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:43.674384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1010/2044] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:43.678692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b664b-b0ec-4ac9-aed6-68fd8ff54593 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:43.688531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b664b-b0ec-4ac9-aed6-68fd8ff54593 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:43.693932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cf318-5dfe-4c59-bab5-0820209cf2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:43.694580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cf318-5dfe-4c59-bab5-0820209cf2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:43.694810 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cf318-5dfe-4c59-bab5-0820209cf2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:43.695079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cf318-5dfe-4c59-bab5-0820209cf2b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:43.695718 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cf318-5dfe-4c59-bab5-0820209cf2b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:31:43.696112 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cf318-5dfe-4c59-bab5-0820209cf2b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:43.697705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cf318-5dfe-4c59-bab5-0820209cf2b4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:43.709809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1035/2045] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:31:43.714157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be6945-1ec6-4841-a2b3-9f6808df9569 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:43.723782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be6945-1ec6-4841-a2b3-9f6808df9569 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:43.728682 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4fda2-b06c-42e4-9319-ea456bc649c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:43.729280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4fda2-b06c-42e4-9319-ea456bc649c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:43.729680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4fda2-b06c-42e4-9319-ea456bc649c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:43.729960 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4fda2-b06c-42e4-9319-ea456bc649c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:43.730602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4fda2-b06c-42e4-9319-ea456bc649c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:43.732567 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4fda2-b06c-42e4-9319-ea456bc649c5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:43.760347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1011/2046] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:31:43 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:31:43.764721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cf318-5dfe-4c59-bab5-0820209cf2b4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:43.774732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cf318-5dfe-4c59-bab5-0820209cf2b4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:43.795127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:43.833946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c91a6-10bc-4a11-82ce-6040b3163d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:43.834558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c91a6-10bc-4a11-82ce-6040b3163d6f admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:43.834841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c91a6-10bc-4a11-82ce-6040b3163d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:43.835117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c91a6-10bc-4a11-82ce-6040b3163d6f admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:43.835756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c91a6-10bc-4a11-82ce-6040b3163d6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:43.837717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c91a6-10bc-4a11-82ce-6040b3163d6f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:43.857891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1036/2047] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:31:43 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:31:43.862284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4fda2-b06c-42e4-9319-ea456bc649c5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:43.873286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4fda2-b06c-42e4-9319-ea456bc649c5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:43.879923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dade7-4594-421c-a007-8d753e4643e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:43.880582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dade7-4594-421c-a007-8d753e4643e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:43.880892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dade7-4594-421c-a007-8d753e4643e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:43.881207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dade7-4594-421c-a007-8d753e4643e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:43.881842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dade7-4594-421c-a007-8d753e4643e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:43.883890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dade7-4594-421c-a007-8d753e4643e6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:43.894967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1012/2048] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:43.899187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c91a6-10bc-4a11-82ce-6040b3163d6f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:43.911469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c91a6-10bc-4a11-82ce-6040b3163d6f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:43.921477 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b92cb-f45c-419a-8561-2747d7a303e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:43.922199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b92cb-f45c-419a-8561-2747d7a303e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:43.922553 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b92cb-f45c-419a-8561-2747d7a303e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:43.923003 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b92cb-f45c-419a-8561-2747d7a303e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:43.923769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b92cb-f45c-419a-8561-2747d7a303e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:43.925696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b92cb-f45c-419a-8561-2747d7a303e3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:43.937139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1037/2049] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:43.941475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dade7-4594-421c-a007-8d753e4643e6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:43.951461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dade7-4594-421c-a007-8d753e4643e6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:43.956660 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e39f4-439a-4891-8495-b2adaa10393d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:43.957316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e39f4-439a-4891-8495-b2adaa10393d admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:43.957598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e39f4-439a-4891-8495-b2adaa10393d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:43.957895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e39f4-439a-4891-8495-b2adaa10393d admin admin] PATH_INFO: `/v3/projects/7f156445785f47fa933ab59c057b1919/users/f43b9cec1b92402e996984d3bf23a9a8/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:43.973001 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e39f4-439a-4891-8495-b2adaa10393d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f156445785f47fa933ab59c057b1919, user_id=f43b9cec1b92402e996984d3bf23a9a8, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:43.975695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e39f4-439a-4891-8495-b2adaa10393d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:43.992340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1013/2050] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:31:43 2022] PUT /identity/v3/projects/7f156445785f47fa933ab59c057b1919/users/f43b9cec1b92402e996984d3bf23a9a8/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:44.003823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cdf65-5da4-4e95-a880-a99ba643f78c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:44.004148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cdf65-5da4-4e95-a880-a99ba643f78c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:44.004429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cdf65-5da4-4e95-a880-a99ba643f78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:44.038139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-534cdf65-5da4-4e95-a880-a99ba643f78c None None] MFA Rules not processed for user `f43b9cec1b92402e996984d3bf23a9a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:44.040335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-534cdf65-5da4-4e95-a880-a99ba643f78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:44.109981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1038/2051] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:44.115130 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e39f4-439a-4891-8495-b2adaa10393d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:44.177433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e39f4-439a-4891-8495-b2adaa10393d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:44.182016 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4f63c-0a51-4a90-883b-c9e876bb5c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:44.182617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4f63c-0a51-4a90-883b-c9e876bb5c77 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:44.182898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4f63c-0a51-4a90-883b-c9e876bb5c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:44.183175 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4f63c-0a51-4a90-883b-c9e876bb5c77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:44.183811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4f63c-0a51-4a90-883b-c9e876bb5c77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:44.185751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4f63c-0a51-4a90-883b-c9e876bb5c77 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:44.211232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1014/2052] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:31:44 2022] POST /identity/v3/projects => generated 393 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:31:44.215350 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cdf65-5da4-4e95-a880-a99ba643f78c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:44.226451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cdf65-5da4-4e95-a880-a99ba643f78c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:44.285636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb5279-4595-4bb5-9dcc-9d164239292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:44.286246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb5279-4595-4bb5-9dcc-9d164239292a admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:44.286660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb5279-4595-4bb5-9dcc-9d164239292a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:44.286937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb5279-4595-4bb5-9dcc-9d164239292a admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:44.287565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb5279-4595-4bb5-9dcc-9d164239292a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:44.289533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb5279-4595-4bb5-9dcc-9d164239292a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:44.316128 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1039/2053] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:31:44 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:31:44.317584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4f63c-0a51-4a90-883b-c9e876bb5c77 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:44.328071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4f63c-0a51-4a90-883b-c9e876bb5c77 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:44.333446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f06bb-30b8-4495-b9cb-9197816fed2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:44.334071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f06bb-30b8-4495-b9cb-9197816fed2b admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:44.334332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f06bb-30b8-4495-b9cb-9197816fed2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:44.334631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f06bb-30b8-4495-b9cb-9197816fed2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:44.335270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f06bb-30b8-4495-b9cb-9197816fed2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:44.337318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9f06bb-30b8-4495-b9cb-9197816fed2b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:44.349578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1015/2054] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:44.353813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb5279-4595-4bb5-9dcc-9d164239292a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:44.364474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb5279-4595-4bb5-9dcc-9d164239292a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:44.369295 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b787cbe-c02c-4507-8ae5-2442b5975c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:44.369957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b787cbe-c02c-4507-8ae5-2442b5975c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:44.370243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b787cbe-c02c-4507-8ae5-2442b5975c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:44.370517 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b787cbe-c02c-4507-8ae5-2442b5975c23 admin admin] PATH_INFO: `/v3/projects/ff6e7502c17345ce95e55225a7474cab/users/0532b7592c5f42ec92bf70e1fab26590/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:44.386095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b787cbe-c02c-4507-8ae5-2442b5975c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff6e7502c17345ce95e55225a7474cab, user_id=0532b7592c5f42ec92bf70e1fab26590, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:44.388898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b787cbe-c02c-4507-8ae5-2442b5975c23 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:44.397732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:44.404378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1040/2055] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:31:44 2022] PUT /identity/v3/projects/ff6e7502c17345ce95e55225a7474cab/users/0532b7592c5f42ec92bf70e1fab26590/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:44.411951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f06bb-30b8-4495-b9cb-9197816fed2b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:44.422263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f06bb-30b8-4495-b9cb-9197816fed2b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:44.449550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:44.481738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34a4f1-3e42-42aa-8fc0-99b5743b3229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:44.482276 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34a4f1-3e42-42aa-8fc0-99b5743b3229 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:44.482555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34a4f1-3e42-42aa-8fc0-99b5743b3229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:44.482824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34a4f1-3e42-42aa-8fc0-99b5743b3229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:44.483457 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc34a4f1-3e42-42aa-8fc0-99b5743b3229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:44.485093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc34a4f1-3e42-42aa-8fc0-99b5743b3229 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:44.495387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1016/2056] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:44.499729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b787cbe-c02c-4507-8ae5-2442b5975c23 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:44.510333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b787cbe-c02c-4507-8ae5-2442b5975c23 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:44.514846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08aca14-1e36-4704-85a6-817d52d1df6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:44.515491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08aca14-1e36-4704-85a6-817d52d1df6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:44.515771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08aca14-1e36-4704-85a6-817d52d1df6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:44.516080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08aca14-1e36-4704-85a6-817d52d1df6a admin admin] PATH_INFO: `/v3/projects/ff6e7502c17345ce95e55225a7474cab/users/0532b7592c5f42ec92bf70e1fab26590/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:44.518141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08aca14-1e36-4704-85a6-817d52d1df6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff6e7502c17345ce95e55225a7474cab, user_id=0532b7592c5f42ec92bf70e1fab26590, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:44.520852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08aca14-1e36-4704-85a6-817d52d1df6a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:44.530748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1041/2057] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:31:44 2022] PUT /identity/v3/projects/ff6e7502c17345ce95e55225a7474cab/users/0532b7592c5f42ec92bf70e1fab26590/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:44.542876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed90a4-cceb-4641-944e-0909c18d1d38 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:44.543269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed90a4-cceb-4641-944e-0909c18d1d38 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:44.543557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed90a4-cceb-4641-944e-0909c18d1d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:44.571822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-baed90a4-cceb-4641-944e-0909c18d1d38 None None] MFA Rules not processed for user `0532b7592c5f42ec92bf70e1fab26590`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:44.573055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-baed90a4-cceb-4641-944e-0909c18d1d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:44.640763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1017/2058] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:44 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:44.653124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08aca14-1e36-4704-85a6-817d52d1df6a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:44.655871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b08aca14-1e36-4704-85a6-817d52d1df6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:44.714365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08aca14-1e36-4704-85a6-817d52d1df6a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:44.718181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539570c-efc5-42fe-9d18-a78af2f0f1e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:44.718783 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539570c-efc5-42fe-9d18-a78af2f0f1e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:44.719061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539570c-efc5-42fe-9d18-a78af2f0f1e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:44.719335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539570c-efc5-42fe-9d18-a78af2f0f1e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:44.729155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539570c-efc5-42fe-9d18-a78af2f0f1e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:44.730666 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539570c-efc5-42fe-9d18-a78af2f0f1e8 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:44.740177 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1042/2059] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:44 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:31:44.789923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed90a4-cceb-4641-944e-0909c18d1d38 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:44.791345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-baed90a4-cceb-4641-944e-0909c18d1d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:44.804033 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed90a4-cceb-4641-944e-0909c18d1d38 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:44.807853 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1d313-1eb0-4e76-aa55-bf614cf6337d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:44.808407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1d313-1eb0-4e76-aa55-bf614cf6337d service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:44.808691 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1d313-1eb0-4e76-aa55-bf614cf6337d service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:44.808966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1d313-1eb0-4e76-aa55-bf614cf6337d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:44.865528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1d313-1eb0-4e76-aa55-bf614cf6337d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:44.866997 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1d313-1eb0-4e76-aa55-bf614cf6337d service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:44.875839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1018/2060] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:44 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:31:49.832026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5d50d-2f9c-47d9-b9a1-f11a8469cee9 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:49.832523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5d50d-2f9c-47d9-b9a1-f11a8469cee9 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:49.832602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5d50d-2f9c-47d9-b9a1-f11a8469cee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:49.853463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-17a5d50d-2f9c-47d9-b9a1-f11a8469cee9 None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:49.854321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-17a5d50d-2f9c-47d9-b9a1-f11a8469cee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:49.920489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1043/2061] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 05:31:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:49.981888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1d313-1eb0-4e76-aa55-bf614cf6337d service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:49.983417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-25d1d313-1eb0-4e76-aa55-bf614cf6337d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:50.037618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:50.058055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1d313-1eb0-4e76-aa55-bf614cf6337d service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:50.062988 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e298b9-d3e7-44cf-b68c-df6a7f178372 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:50.063595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e298b9-d3e7-44cf-b68c-df6a7f178372 service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:50.063871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e298b9-d3e7-44cf-b68c-df6a7f178372 service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:50.064288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e298b9-d3e7-44cf-b68c-df6a7f178372 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:50.073560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e298b9-d3e7-44cf-b68c-df6a7f178372 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:50.075151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e298b9-d3e7-44cf-b68c-df6a7f178372 service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:50.084064 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1019/2062] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:31:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 05:31:50.494172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5d50d-2f9c-47d9-b9a1-f11a8469cee9 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:50.584751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5d50d-2f9c-47d9-b9a1-f11a8469cee9 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:50.593086 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f926549-8c5e-4fab-9650-9fc0a784d3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:50.594426 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f926549-8c5e-4fab-9650-9fc0a784d3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:50.594701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f926549-8c5e-4fab-9650-9fc0a784d3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:50.594980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f926549-8c5e-4fab-9650-9fc0a784d3cd admin admin] PATH_INFO: `/v3/users/f43b9cec1b92402e996984d3bf23a9a8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:50.598046 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f926549-8c5e-4fab-9650-9fc0a784d3cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f43b9cec1b92402e996984d3bf23a9a8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:50.603134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f926549-8c5e-4fab-9650-9fc0a784d3cd admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:50.669971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:50.707076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4f926549-8c5e-4fab-9650-9fc0a784d3cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f43b9cec1b92402e996984d3bf23a9a8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:50.708085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4f926549-8c5e-4fab-9650-9fc0a784d3cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f43b9cec1b92402e996984d3bf23a9a8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:50.716906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4f926549-8c5e-4fab-9650-9fc0a784d3cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f43b9cec1b92402e996984d3bf23a9a8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:50.748763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4f926549-8c5e-4fab-9650-9fc0a784d3cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f43b9cec1b92402e996984d3bf23a9a8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:50.780928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4f926549-8c5e-4fab-9650-9fc0a784d3cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f43b9cec1b92402e996984d3bf23a9a8'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:50.805930 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1044/2063] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:31:50 2022] DELETE /identity/v3/users/f43b9cec1b92402e996984d3bf23a9a8 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:50.816895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cae75-f9d6-49a8-9ac7-83e596627531 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:50.817108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cae75-f9d6-49a8-9ac7-83e596627531 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:50.817382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cae75-f9d6-49a8-9ac7-83e596627531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:50.849446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-126cae75-f9d6-49a8-9ac7-83e596627531 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:50.851347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-126cae75-f9d6-49a8-9ac7-83e596627531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:50.865709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62803d-41b9-41b1-bd9a-4a9ce46980c2 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:50.867006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62803d-41b9-41b1-bd9a-4a9ce46980c2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:50.867386 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62803d-41b9-41b1-bd9a-4a9ce46980c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:50.889662 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7c62803d-41b9-41b1-bd9a-4a9ce46980c2 None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:50.892015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7c62803d-41b9-41b1-bd9a-4a9ce46980c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:50.924470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1020/2064] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:50.934700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cae75-f9d6-49a8-9ac7-83e596627531 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:50.936733 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-126cae75-f9d6-49a8-9ac7-83e596627531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:50.992958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1045/2065] 10.222.0.33 () {58 vars in 967 bytes} [Sun Sep 11 05:31:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:51.007726 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62803d-41b9-41b1-bd9a-4a9ce46980c2 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:51.010329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7c62803d-41b9-41b1-bd9a-4a9ce46980c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:51.014338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cae75-f9d6-49a8-9ac7-83e596627531 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:51.020362 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539b6e7-3c99-4b39-ac7f-f7eaf6ca243a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:51.021466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539b6e7-3c99-4b39-ac7f-f7eaf6ca243a service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:51.021907 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539b6e7-3c99-4b39-ac7f-f7eaf6ca243a service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:51.022824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539b6e7-3c99-4b39-ac7f-f7eaf6ca243a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:51.045926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539b6e7-3c99-4b39-ac7f-f7eaf6ca243a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:51.047620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539b6e7-3c99-4b39-ac7f-f7eaf6ca243a service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:51.060199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1021/2066] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:31:51.101460 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62803d-41b9-41b1-bd9a-4a9ce46980c2 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:51.107183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8790c79-e110-4a27-86fc-ef16855e983d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:51.107811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8790c79-e110-4a27-86fc-ef16855e983d service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:51.108125 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8790c79-e110-4a27-86fc-ef16855e983d service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:51.108403 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8790c79-e110-4a27-86fc-ef16855e983d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:51.119694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8790c79-e110-4a27-86fc-ef16855e983d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:51.121449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8790c79-e110-4a27-86fc-ef16855e983d service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:51.132482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1046/2067] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:31:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 05:31:51.347419 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e837953-f649-4bbd-add7-d63a9336887e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:51.347747 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e837953-f649-4bbd-add7-d63a9336887e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:51.348034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e837953-f649-4bbd-add7-d63a9336887e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:51.397576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8e837953-f649-4bbd-add7-d63a9336887e None None] MFA Rules not processed for user `8424e74e566d4bf3b8840b5d5c20aa1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:51.400029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8e837953-f649-4bbd-add7-d63a9336887e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:51.411972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8790c79-e110-4a27-86fc-ef16855e983d service swift] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:51.413646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b8790c79-e110-4a27-86fc-ef16855e983d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:51.441346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8790c79-e110-4a27-86fc-ef16855e983d service swift] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:51.447529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:51.448437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:51.448903 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:51.449387 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] PATH_INFO: `/v3/projects/7f156445785f47fa933ab59c057b1919` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:51.450769 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f156445785f47fa933ab59c057b1919)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:51.462102 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:51.492269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1022/2068] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:31:51.520372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:51.531864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f156445785f47fa933ab59c057b1919', 'request_id': 'req-3612f95b-b89c-4158-a585-9c6c3a61909d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:51.541269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f156445785f47fa933ab59c057b1919', 'request_id': 'req-3612f95b-b89c-4158-a585-9c6c3a61909d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:51.547067 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189272ee-ac28-4795-9d7f-cbca977620f8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:51.547376 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189272ee-ac28-4795-9d7f-cbca977620f8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:51.547628 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189272ee-ac28-4795-9d7f-cbca977620f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:51.550247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f156445785f47fa933ab59c057b1919', 'request_id': 'req-3612f95b-b89c-4158-a585-9c6c3a61909d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:51.583944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] The token cache is being invalidate because project 7f156445785f47fa933ab59c057b1919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:31:51.584523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:51.585213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:51.586764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1047/2069] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:31:51 2022] DELETE /identity/v3/projects/7f156445785f47fa933ab59c057b1919 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:31:51.591458 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:51.592963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:51.598971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-189272ee-ac28-4795-9d7f-cbca977620f8 None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:51.601729 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-189272ee-ac28-4795-9d7f-cbca977620f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:51.631559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:51.673574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612f95b-b89c-4158-a585-9c6c3a61909d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:51.680859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5315d-8214-4544-8463-a6e17e46c251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:51.681641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5315d-8214-4544-8463-a6e17e46c251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:51.682001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5315d-8214-4544-8463-a6e17e46c251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:51.682409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5315d-8214-4544-8463-a6e17e46c251 admin admin] PATH_INFO: `/v3/users/0532b7592c5f42ec92bf70e1fab26590` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:51.683937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5315d-8214-4544-8463-a6e17e46c251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0532b7592c5f42ec92bf70e1fab26590)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:51.686979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5315d-8214-4544-8463-a6e17e46c251 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:51.690935 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1023/2070] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 05:31:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:51.707306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189272ee-ac28-4795-9d7f-cbca977620f8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:51.709301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-189272ee-ac28-4795-9d7f-cbca977620f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:51.753652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-87c5315d-8214-4544-8463-a6e17e46c251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0532b7592c5f42ec92bf70e1fab26590'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:51.755497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-87c5315d-8214-4544-8463-a6e17e46c251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0532b7592c5f42ec92bf70e1fab26590'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:51.765597 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-87c5315d-8214-4544-8463-a6e17e46c251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0532b7592c5f42ec92bf70e1fab26590'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:51.777751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-87c5315d-8214-4544-8463-a6e17e46c251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0532b7592c5f42ec92bf70e1fab26590'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:51.797267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-87c5315d-8214-4544-8463-a6e17e46c251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0532b7592c5f42ec92bf70e1fab26590'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:51.806181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1048/2071] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:31:51 2022] DELETE /identity/v3/users/0532b7592c5f42ec92bf70e1fab26590 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:51.852840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189272ee-ac28-4795-9d7f-cbca977620f8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:51.977543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f620be-c522-4291-abad-c1238683f970 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:51.978349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f620be-c522-4291-abad-c1238683f970 service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:51.979056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f620be-c522-4291-abad-c1238683f970 service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:51.979338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f620be-c522-4291-abad-c1238683f970 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:51.981829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-74f620be-c522-4291-abad-c1238683f970 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:52.083886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f620be-c522-4291-abad-c1238683f970 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:52.086519 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f620be-c522-4291-abad-c1238683f970 service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:52.104585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1024/2072] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:31:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 05:31:52.597276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6da6d6-db42-4b38-ab99-d95c2d6bb8f6 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:52.598430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6da6d6-db42-4b38-ab99-d95c2d6bb8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:52.598898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6da6d6-db42-4b38-ab99-d95c2d6bb8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:52.640631 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1c6da6d6-db42-4b38-ab99-d95c2d6bb8f6 None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:52.643665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1c6da6d6-db42-4b38-ab99-d95c2d6bb8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:52.659681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f620be-c522-4291-abad-c1238683f970 service swift] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:52.662527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-74f620be-c522-4291-abad-c1238683f970 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:52.668742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1049/2073] 10.222.0.33 () {58 vars in 967 bytes} [Sun Sep 11 05:31:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:31:52.683543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6da6d6-db42-4b38-ab99-d95c2d6bb8f6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:52.686145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1c6da6d6-db42-4b38-ab99-d95c2d6bb8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:52.705003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:52.737505 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6da6d6-db42-4b38-ab99-d95c2d6bb8f6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:52.753947 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d5f49-6812-49f8-ade4-25549642bbe4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:52.754620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d5f49-6812-49f8-ade4-25549642bbe4 service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:52.754903 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d5f49-6812-49f8-ade4-25549642bbe4 service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:52.755175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d5f49-6812-49f8-ade4-25549642bbe4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:52.775901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d5f49-6812-49f8-ade4-25549642bbe4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:52.777371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d5f49-6812-49f8-ade4-25549642bbe4 service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:52.790709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1050/2074] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:31:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 05:31:52.836516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f620be-c522-4291-abad-c1238683f970 service swift] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:52.841893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:52.842543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:52.843127 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:52.843402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] PATH_INFO: `/v3/projects/ff6e7502c17345ce95e55225a7474cab` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:52.845182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff6e7502c17345ce95e55225a7474cab)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:52.848341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:52.918571 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff6e7502c17345ce95e55225a7474cab', 'request_id': 'req-9de21a04-1797-463d-a21e-fa75814f4e5a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:52.930290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff6e7502c17345ce95e55225a7474cab', 'request_id': 'req-9de21a04-1797-463d-a21e-fa75814f4e5a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:52.945098 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff6e7502c17345ce95e55225a7474cab', 'request_id': 'req-9de21a04-1797-463d-a21e-fa75814f4e5a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:52.965721 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] The token cache is being invalidate because project ff6e7502c17345ce95e55225a7474cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:31:52.966249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:52.967822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:52.973221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1025/2075] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:31:52 2022] DELETE /identity/v3/projects/ff6e7502c17345ce95e55225a7474cab => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:52.978890 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0c15b-572e-413a-8f28-decdce7fcf48 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:52.979193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0c15b-572e-413a-8f28-decdce7fcf48 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:52.979469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0c15b-572e-413a-8f28-decdce7fcf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:53.008897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3da0c15b-572e-413a-8f28-decdce7fcf48 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:53.010530 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3da0c15b-572e-413a-8f28-decdce7fcf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:53.054189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:53.088535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:53.090792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:53.116758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1051/2076] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:53.120814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2397e7-48a4-45fe-b5b7-3563b7bf6c58 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:53.121010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2397e7-48a4-45fe-b5b7-3563b7bf6c58 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:53.121483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2397e7-48a4-45fe-b5b7-3563b7bf6c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:53.148626 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-7e2397e7-48a4-45fe-b5b7-3563b7bf6c58 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:53.149865 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7e2397e7-48a4-45fe-b5b7-3563b7bf6c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:53.159927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1052/2077] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:53.166934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2397e7-48a4-45fe-b5b7-3563b7bf6c58 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:53.184080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2397e7-48a4-45fe-b5b7-3563b7bf6c58 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:53.193328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da7887-91b5-4779-a12a-47594de04170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:53.194149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da7887-91b5-4779-a12a-47594de04170 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:53.194516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da7887-91b5-4779-a12a-47594de04170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:53.194810 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da7887-91b5-4779-a12a-47594de04170 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:53.195438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da7887-91b5-4779-a12a-47594de04170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:31:53.195741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da7887-91b5-4779-a12a-47594de04170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:53.197641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da7887-91b5-4779-a12a-47594de04170 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:53.213959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1053/2078] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:31:53.220028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da7887-91b5-4779-a12a-47594de04170 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:53.220809 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de21a04-1797-463d-a21e-fa75814f4e5a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:53.229765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9e5f4-5bb0-42b8-b01e-e7638f9361ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:53.230504 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9e5f4-5bb0-42b8-b01e-e7638f9361ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:53.230862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9e5f4-5bb0-42b8-b01e-e7638f9361ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:53.231268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9e5f4-5bb0-42b8-b01e-e7638f9361ce admin admin] PATH_INFO: `/v3/users/2e89e554b1854c1a902e6962d8103dca` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:53.232728 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e9e5f4-5bb0-42b8-b01e-e7638f9361ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e89e554b1854c1a902e6962d8103dca)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:53.236296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e9e5f4-5bb0-42b8-b01e-e7638f9361ce admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:53.259123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da7887-91b5-4779-a12a-47594de04170 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:53.273445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60520fc4-0c9a-4eac-9695-5d0688082219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:53.275199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60520fc4-0c9a-4eac-9695-5d0688082219 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:53.275199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60520fc4-0c9a-4eac-9695-5d0688082219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:53.275817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60520fc4-0c9a-4eac-9695-5d0688082219 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:53.277108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60520fc4-0c9a-4eac-9695-5d0688082219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:53.284770 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60520fc4-0c9a-4eac-9695-5d0688082219 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:53.326463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-29e9e5f4-5bb0-42b8-b01e-e7638f9361ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e89e554b1854c1a902e6962d8103dca'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:53.328132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:53.339394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-29e9e5f4-5bb0-42b8-b01e-e7638f9361ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e89e554b1854c1a902e6962d8103dca'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:53.358610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-29e9e5f4-5bb0-42b8-b01e-e7638f9361ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e89e554b1854c1a902e6962d8103dca'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:53.359625 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-29e9e5f4-5bb0-42b8-b01e-e7638f9361ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e89e554b1854c1a902e6962d8103dca'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:53.360510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-29e9e5f4-5bb0-42b8-b01e-e7638f9361ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e89e554b1854c1a902e6962d8103dca'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:53.365515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1054/2079] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:31:53 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:31:53.369852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60520fc4-0c9a-4eac-9695-5d0688082219 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:53.372448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-60520fc4-0c9a-4eac-9695-5d0688082219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:53.383425 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1026/2080] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:31:53 2022] DELETE /identity/v3/users/2e89e554b1854c1a902e6962d8103dca => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:31:53.388673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a13bf-8227-4a61-9b2d-e078e1e74d9b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:53.389148 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a13bf-8227-4a61-9b2d-e078e1e74d9b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:53.389565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a13bf-8227-4a61-9b2d-e078e1e74d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:53.426688 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-6b2a13bf-8227-4a61-9b2d-e078e1e74d9b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:53.428796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6b2a13bf-8227-4a61-9b2d-e078e1e74d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:53.476508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60520fc4-0c9a-4eac-9695-5d0688082219 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:53.527698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1027/2081] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:31:53.539099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a13bf-8227-4a61-9b2d-e078e1e74d9b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:53.541355 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6b2a13bf-8227-4a61-9b2d-e078e1e74d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:53.559876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3511395-ac28-4485-8782-ce7c704324f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:53.560676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3511395-ac28-4485-8782-ce7c704324f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:53.561073 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3511395-ac28-4485-8782-ce7c704324f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:53.561367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3511395-ac28-4485-8782-ce7c704324f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:53.562492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3511395-ac28-4485-8782-ce7c704324f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:53.565332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3511395-ac28-4485-8782-ce7c704324f0 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:53.597713 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1055/2082] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:31:53 2022] POST /identity/v3/users => generated 291 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:31:53.602469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3511395-ac28-4485-8782-ce7c704324f0 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:53.624185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3511395-ac28-4485-8782-ce7c704324f0 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:53.629571 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457971f4-7786-41cc-828c-267d17bd51c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:53.629732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a13bf-8227-4a61-9b2d-e078e1e74d9b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:53.630262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457971f4-7786-41cc-828c-267d17bd51c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:53.630545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457971f4-7786-41cc-828c-267d17bd51c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:53.630925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457971f4-7786-41cc-828c-267d17bd51c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:53.631637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457971f4-7786-41cc-828c-267d17bd51c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:53.633268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457971f4-7786-41cc-828c-267d17bd51c4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:53.634341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb2020-2dc7-4a7e-8f83-367856dc36fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:53.635080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb2020-2dc7-4a7e-8f83-367856dc36fb service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:53.635289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb2020-2dc7-4a7e-8f83-367856dc36fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:53.635804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb2020-2dc7-4a7e-8f83-367856dc36fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:53.649556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1056/2083] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:53.654357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457971f4-7786-41cc-828c-267d17bd51c4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:53.656295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb2020-2dc7-4a7e-8f83-367856dc36fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:53.658484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb2020-2dc7-4a7e-8f83-367856dc36fb service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:53.670168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457971f4-7786-41cc-828c-267d17bd51c4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:53.672066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1028/2084] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:31:53.675767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9474c3-803f-451b-ac44-d351ea566b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:53.676548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9474c3-803f-451b-ac44-d351ea566b78 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:53.676900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9474c3-803f-451b-ac44-d351ea566b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:53.677246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9474c3-803f-451b-ac44-d351ea566b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:53.678012 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9474c3-803f-451b-ac44-d351ea566b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:53.679782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9474c3-803f-451b-ac44-d351ea566b78 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:53.694983 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1057/2085] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:53.700454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb2020-2dc7-4a7e-8f83-367856dc36fb service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:53.712263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb2020-2dc7-4a7e-8f83-367856dc36fb service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:53.717741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b313b9-e661-496e-bd5d-0d796377e5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:53.718819 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b313b9-e661-496e-bd5d-0d796377e5e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:53.719104 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b313b9-e661-496e-bd5d-0d796377e5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:53.719377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b313b9-e661-496e-bd5d-0d796377e5e7 admin admin] PATH_INFO: `/v3/projects/a647b9e2276048ecb6ec88d50f4e405f/users/9a20e97b5b4f49ddbe0a2f51b49999a0/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:53.761454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b313b9-e661-496e-bd5d-0d796377e5e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a647b9e2276048ecb6ec88d50f4e405f, user_id=9a20e97b5b4f49ddbe0a2f51b49999a0, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:53.765188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b313b9-e661-496e-bd5d-0d796377e5e7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:53.785839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1029/2086] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:31:53 2022] PUT /identity/v3/projects/a647b9e2276048ecb6ec88d50f4e405f/users/9a20e97b5b4f49ddbe0a2f51b49999a0/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:53.796602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247cd92-7a8d-4353-a11a-e4f990ee1538 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:53.796804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247cd92-7a8d-4353-a11a-e4f990ee1538 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:53.797015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247cd92-7a8d-4353-a11a-e4f990ee1538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:53.811672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:53.843319 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9247cd92-7a8d-4353-a11a-e4f990ee1538 None None] MFA Rules not processed for user `9a20e97b5b4f49ddbe0a2f51b49999a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:53.845831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9247cd92-7a8d-4353-a11a-e4f990ee1538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:53.941048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b313b9-e661-496e-bd5d-0d796377e5e7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:53.946564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1058/2087] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:54.001108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:54.024557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b313b9-e661-496e-bd5d-0d796377e5e7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.029387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622012-927d-46b5-821e-ce67602fb45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.029591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622012-927d-46b5-821e-ce67602fb45a admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.030026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622012-927d-46b5-821e-ce67602fb45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.030376 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622012-927d-46b5-821e-ce67602fb45a admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.031145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69622012-927d-46b5-821e-ce67602fb45a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.034741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69622012-927d-46b5-821e-ce67602fb45a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.073644 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1030/2088] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:31:53 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:31:54.078458 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247cd92-7a8d-4353-a11a-e4f990ee1538 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:54.089287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622012-927d-46b5-821e-ce67602fb45a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:54.091326 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-69622012-927d-46b5-821e-ce67602fb45a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:54.099104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247cd92-7a8d-4353-a11a-e4f990ee1538 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.190455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622012-927d-46b5-821e-ce67602fb45a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.197105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556da41-34d2-43af-bc75-cd043234a52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.197952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556da41-34d2-43af-bc75-cd043234a52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.198246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556da41-34d2-43af-bc75-cd043234a52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.198521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556da41-34d2-43af-bc75-cd043234a52c admin admin] PATH_INFO: `/v3/projects/d88908a970db4ee8a6ad8586538306f2` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.199883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556da41-34d2-43af-bc75-cd043234a52c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d88908a970db4ee8a6ad8586538306f2)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.201755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8e03f-e205-44f9-8576-5cf9551cc822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.201904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556da41-34d2-43af-bc75-cd043234a52c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.202559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8e03f-e205-44f9-8576-5cf9551cc822 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.203001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8e03f-e205-44f9-8576-5cf9551cc822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.203268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8e03f-e205-44f9-8576-5cf9551cc822 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.205702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8e03f-e205-44f9-8576-5cf9551cc822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.207886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8e03f-e205-44f9-8576-5cf9551cc822 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.240671 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1059/2089] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:31:54 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:31:54.248133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8e03f-e205-44f9-8576-5cf9551cc822 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:54.264033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8e03f-e205-44f9-8576-5cf9551cc822 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.280993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e556da41-34d2-43af-bc75-cd043234a52c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd88908a970db4ee8a6ad8586538306f2', 'request_id': 'req-e556da41-34d2-43af-bc75-cd043234a52c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:54.291658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e556da41-34d2-43af-bc75-cd043234a52c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd88908a970db4ee8a6ad8586538306f2', 'request_id': 'req-e556da41-34d2-43af-bc75-cd043234a52c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:54.300647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e556da41-34d2-43af-bc75-cd043234a52c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd88908a970db4ee8a6ad8586538306f2', 'request_id': 'req-e556da41-34d2-43af-bc75-cd043234a52c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:54.315186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e556da41-34d2-43af-bc75-cd043234a52c admin admin] The token cache is being invalidate because project d88908a970db4ee8a6ad8586538306f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:31:54.315859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e556da41-34d2-43af-bc75-cd043234a52c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:54.316944 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e556da41-34d2-43af-bc75-cd043234a52c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:54.318811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1031/2090] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:31:54 2022] DELETE /identity/v3/projects/d88908a970db4ee8a6ad8586538306f2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:31:54.323836 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c56bf-07d1-4eba-bcd6-b06000b1f86a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.324318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c56bf-07d1-4eba-bcd6-b06000b1f86a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.324710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c56bf-07d1-4eba-bcd6-b06000b1f86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.358871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df5ea1-9284-45f6-a723-6953f011e7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.359899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-506c56bf-07d1-4eba-bcd6-b06000b1f86a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:54.361809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df5ea1-9284-45f6-a723-6953f011e7ca admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.362189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-506c56bf-07d1-4eba-bcd6-b06000b1f86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:54.362428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df5ea1-9284-45f6-a723-6953f011e7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.362712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df5ea1-9284-45f6-a723-6953f011e7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.363445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df5ea1-9284-45f6-a723-6953f011e7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.365673 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df5ea1-9284-45f6-a723-6953f011e7ca admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.372624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1032/2091] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:54.379689 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1060/2092] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:31:54.384103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a81f0-2e55-45dd-a7f4-b1b0c817758b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.384103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a81f0-2e55-45dd-a7f4-b1b0c817758b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.384103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a81f0-2e55-45dd-a7f4-b1b0c817758b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.385760 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df5ea1-9284-45f6-a723-6953f011e7ca admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:54.392793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-77df5ea1-9284-45f6-a723-6953f011e7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:54.407172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df5ea1-9284-45f6-a723-6953f011e7ca admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.412196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fc567-ab7c-4e4f-8aaa-5ea4062e6161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.412856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fc567-ab7c-4e4f-8aaa-5ea4062e6161 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.413139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fc567-ab7c-4e4f-8aaa-5ea4062e6161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.413419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fc567-ab7c-4e4f-8aaa-5ea4062e6161 admin admin] PATH_INFO: `/v3/projects/d8215d6ffeba47c89f17810d2b826d57/users/7c2e23e2869b45c98f003dbc2c0527fc/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.417143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0a2a81f0-2e55-45dd-a7f4-b1b0c817758b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:54.418493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0a2a81f0-2e55-45dd-a7f4-b1b0c817758b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:54.428573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1033/2093] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:54.433713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a81f0-2e55-45dd-a7f4-b1b0c817758b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:54.435631 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fc567-ab7c-4e4f-8aaa-5ea4062e6161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8215d6ffeba47c89f17810d2b826d57, user_id=7c2e23e2869b45c98f003dbc2c0527fc, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.438670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fc567-ab7c-4e4f-8aaa-5ea4062e6161 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.443770 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:54.444042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a81f0-2e55-45dd-a7f4-b1b0c817758b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.451185 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cbc26-d196-45d6-9ff8-c832084c0013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.451795 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cbc26-d196-45d6-9ff8-c832084c0013 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.452197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cbc26-d196-45d6-9ff8-c832084c0013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.452543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cbc26-d196-45d6-9ff8-c832084c0013 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.453273 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169cbc26-d196-45d6-9ff8-c832084c0013 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:31:54.453576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169cbc26-d196-45d6-9ff8-c832084c0013 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.455921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169cbc26-d196-45d6-9ff8-c832084c0013 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.460733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1061/2094] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:31:54 2022] PUT /identity/v3/projects/d8215d6ffeba47c89f17810d2b826d57/users/7c2e23e2869b45c98f003dbc2c0527fc/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:54.465092 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fc567-ab7c-4e4f-8aaa-5ea4062e6161 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:54.469770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1034/2095] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:31:54.474028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cbc26-d196-45d6-9ff8-c832084c0013 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:54.484368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cbc26-d196-45d6-9ff8-c832084c0013 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.522001 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:54.535435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fc567-ab7c-4e4f-8aaa-5ea4062e6161 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.543373 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e22c7-7843-41fe-9dd5-605b89362f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.544486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e22c7-7843-41fe-9dd5-605b89362f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.544677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e22c7-7843-41fe-9dd5-605b89362f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.545169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e22c7-7843-41fe-9dd5-605b89362f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.545990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e22c7-7843-41fe-9dd5-605b89362f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.547974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e22c7-7843-41fe-9dd5-605b89362f1c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.564504 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1062/2096] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:31:54.564700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffb972-ae97-4513-ba88-6d4a3a5ee5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.564700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffb972-ae97-4513-ba88-6d4a3a5ee5c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.564700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffb972-ae97-4513-ba88-6d4a3a5ee5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.564926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffb972-ae97-4513-ba88-6d4a3a5ee5c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.565913 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffb972-ae97-4513-ba88-6d4a3a5ee5c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.566031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e22c7-7843-41fe-9dd5-605b89362f1c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:54.567512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffb972-ae97-4513-ba88-6d4a3a5ee5c2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.575779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e22c7-7843-41fe-9dd5-605b89362f1c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.580864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87db33f-49a5-48b3-88e2-5529fec868cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.581529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87db33f-49a5-48b3-88e2-5529fec868cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.581812 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87db33f-49a5-48b3-88e2-5529fec868cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.582103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87db33f-49a5-48b3-88e2-5529fec868cc admin admin] PATH_INFO: `/v3/projects/d8215d6ffeba47c89f17810d2b826d57/users/7c2e23e2869b45c98f003dbc2c0527fc/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.584059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87db33f-49a5-48b3-88e2-5529fec868cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8215d6ffeba47c89f17810d2b826d57, user_id=7c2e23e2869b45c98f003dbc2c0527fc, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.586759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87db33f-49a5-48b3-88e2-5529fec868cc admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.597325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1035/2097] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:31:54 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:31:54.597690 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1063/2098] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:31:54 2022] PUT /identity/v3/projects/d8215d6ffeba47c89f17810d2b826d57/users/7c2e23e2869b45c98f003dbc2c0527fc/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:54.607453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffb972-ae97-4513-ba88-6d4a3a5ee5c2 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:54.609398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78e168-7491-4474-ad9b-d91bb89328bb None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.609970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78e168-7491-4474-ad9b-d91bb89328bb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.610097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78e168-7491-4474-ad9b-d91bb89328bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.642465 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9b78e168-7491-4474-ad9b-d91bb89328bb None None] MFA Rules not processed for user `7c2e23e2869b45c98f003dbc2c0527fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:54.643704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9b78e168-7491-4474-ad9b-d91bb89328bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:54.679103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffb972-ae97-4513-ba88-6d4a3a5ee5c2 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.684826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f4df2-4528-4af9-a590-fd556918da83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.685540 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f4df2-4528-4af9-a590-fd556918da83 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.685830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f4df2-4528-4af9-a590-fd556918da83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.686100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f4df2-4528-4af9-a590-fd556918da83 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.686730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f4df2-4528-4af9-a590-fd556918da83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.688705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f4df2-4528-4af9-a590-fd556918da83 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.716228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1036/2099] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:31:54 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:31:54.717361 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f4df2-4528-4af9-a590-fd556918da83 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:54.732547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f4df2-4528-4af9-a590-fd556918da83 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.735578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1064/2100] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:31:54.739417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739455d3-b3c8-4d19-9ce6-0765e5e1aa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.739417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739455d3-b3c8-4d19-9ce6-0765e5e1aa3f admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.739417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739455d3-b3c8-4d19-9ce6-0765e5e1aa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.739417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739455d3-b3c8-4d19-9ce6-0765e5e1aa3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.740478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739455d3-b3c8-4d19-9ce6-0765e5e1aa3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.742232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739455d3-b3c8-4d19-9ce6-0765e5e1aa3f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.748018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78e168-7491-4474-ad9b-d91bb89328bb None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:54.750222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9b78e168-7491-4474-ad9b-d91bb89328bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:54.759947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1037/2101] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:54.764544 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739455d3-b3c8-4d19-9ce6-0765e5e1aa3f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:54.775458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739455d3-b3c8-4d19-9ce6-0765e5e1aa3f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.780179 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73052c-d3f7-41ce-b746-0be04e07911d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.780857 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73052c-d3f7-41ce-b746-0be04e07911d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.781383 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73052c-d3f7-41ce-b746-0be04e07911d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.781704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73052c-d3f7-41ce-b746-0be04e07911d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.782349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73052c-d3f7-41ce-b746-0be04e07911d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.784246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73052c-d3f7-41ce-b746-0be04e07911d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.800298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1038/2102] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:31:54.805515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73052c-d3f7-41ce-b746-0be04e07911d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:54.818575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73052c-d3f7-41ce-b746-0be04e07911d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.823452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da5441-eacc-43cf-8e80-9b63fc10c711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.824155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da5441-eacc-43cf-8e80-9b63fc10c711 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.824439 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da5441-eacc-43cf-8e80-9b63fc10c711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.824713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da5441-eacc-43cf-8e80-9b63fc10c711 admin admin] PATH_INFO: `/v3/projects/ebd316863fbf4a548fd793b741599881/users/9902122301ed4431a3ec1276a7af71b2/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.835890 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78e168-7491-4474-ad9b-d91bb89328bb None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:54.839620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d3614-a02f-46bb-b9ba-cd44093732a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:54.840287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d3614-a02f-46bb-b9ba-cd44093732a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.840571 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d3614-a02f-46bb-b9ba-cd44093732a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.840672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da5441-eacc-43cf-8e80-9b63fc10c711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd316863fbf4a548fd793b741599881, user_id=9902122301ed4431a3ec1276a7af71b2, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.840838 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d3614-a02f-46bb-b9ba-cd44093732a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.843286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da5441-eacc-43cf-8e80-9b63fc10c711 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.858654 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1039/2103] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:31:54 2022] PUT /identity/v3/projects/ebd316863fbf4a548fd793b741599881/users/9902122301ed4431a3ec1276a7af71b2/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:31:54.868754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d3614-a02f-46bb-b9ba-cd44093732a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:54.870165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf24540-ce05-4625-909f-c3809e3e44be None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:54.870285 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d3614-a02f-46bb-b9ba-cd44093732a3 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:54.870451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf24540-ce05-4625-909f-c3809e3e44be None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:54.870724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf24540-ce05-4625-909f-c3809e3e44be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:54.881064 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1065/2104] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:31:54.908717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-fcf24540-ce05-4625-909f-c3809e3e44be None None] MFA Rules not processed for user `9902122301ed4431a3ec1276a7af71b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:54.910163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fcf24540-ce05-4625-909f-c3809e3e44be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:54.984488 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:54.996758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d3614-a02f-46bb-b9ba-cd44093732a3 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:55.003245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:55.008126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d3614-a02f-46bb-b9ba-cd44093732a3 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:55.013569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1040/2105] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:31:55.029174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf24540-ce05-4625-909f-c3809e3e44be None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:55.030554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fcf24540-ce05-4625-909f-c3809e3e44be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:55.082141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931fa38-be78-4a15-aef7-01b637b61537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:55.082767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931fa38-be78-4a15-aef7-01b637b61537 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:55.083044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931fa38-be78-4a15-aef7-01b637b61537 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:55.083516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931fa38-be78-4a15-aef7-01b637b61537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:55.087906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1931fa38-be78-4a15-aef7-01b637b61537 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:55.157475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf24540-ce05-4625-909f-c3809e3e44be None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:55.164218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e1b15-bc9d-4596-93c3-117f844e6117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:55.165166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e1b15-bc9d-4596-93c3-117f844e6117 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:55.165402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e1b15-bc9d-4596-93c3-117f844e6117 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:55.165660 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e1b15-bc9d-4596-93c3-117f844e6117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:55.167895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931fa38-be78-4a15-aef7-01b637b61537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:55.169404 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931fa38-be78-4a15-aef7-01b637b61537 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:55.176832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e1b15-bc9d-4596-93c3-117f844e6117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:55.178405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e1b15-bc9d-4596-93c3-117f844e6117 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:55.179188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1066/2106] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:31:55.188580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1041/2107] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:31:55.645847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931fa38-be78-4a15-aef7-01b637b61537 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:55.648577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1931fa38-be78-4a15-aef7-01b637b61537 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:55.741519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931fa38-be78-4a15-aef7-01b637b61537 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:55.746843 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:55.747494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:55.747817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:55.748061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] PATH_INFO: `/v3/users/b67b9e5012f34ab4a17ab579131a986e` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:55.749218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b67b9e5012f34ab4a17ab579131a986e)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:55.752175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:55.813918 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b67b9e5012f34ab4a17ab579131a986e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:55.815127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b67b9e5012f34ab4a17ab579131a986e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:55.823559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b67b9e5012f34ab4a17ab579131a986e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:55.827320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:55.833042 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b67b9e5012f34ab4a17ab579131a986e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:55.849236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b67b9e5012f34ab4a17ab579131a986e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:55.857310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1067/2108] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:31:55 2022] DELETE /identity/v3/users/b67b9e5012f34ab4a17ab579131a986e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:55.865111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec9065-3ddc-4a95-b3b5-e5a2d93d5f84 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:55.865656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec9065-3ddc-4a95-b3b5-e5a2d93d5f84 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:55.866120 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec9065-3ddc-4a95-b3b5-e5a2d93d5f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:55.897512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-81ec9065-3ddc-4a95-b3b5-e5a2d93d5f84 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:55.899116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-81ec9065-3ddc-4a95-b3b5-e5a2d93d5f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:55.969955 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1042/2109] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:55.982860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:55.984563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:56.046099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6819364-2955-4c09-98fe-4f45cb2ab1cb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:56.050029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f5949-0df8-48ea-9a31-08359c4a93e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:56.050658 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f5949-0df8-48ea-9a31-08359c4a93e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:56.050937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f5949-0df8-48ea-9a31-08359c4a93e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:56.051205 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f5949-0df8-48ea-9a31-08359c4a93e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:56.060910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f5949-0df8-48ea-9a31-08359c4a93e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:56.062418 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f5949-0df8-48ea-9a31-08359c4a93e5 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:56.072876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1068/2110] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:31:56.289277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec9065-3ddc-4a95-b3b5-e5a2d93d5f84 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:56.291027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-81ec9065-3ddc-4a95-b3b5-e5a2d93d5f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:56.306318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec9065-3ddc-4a95-b3b5-e5a2d93d5f84 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:56.311451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e0cc2-80b9-4478-b4f6-06924f324b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:56.312051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e0cc2-80b9-4478-b4f6-06924f324b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:56.312465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e0cc2-80b9-4478-b4f6-06924f324b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:56.312593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e0cc2-80b9-4478-b4f6-06924f324b5d admin admin] PATH_INFO: `/v3/projects/2a9ef16092824511b11494d4388e3c58` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:56.314583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e0cc2-80b9-4478-b4f6-06924f324b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a9ef16092824511b11494d4388e3c58)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:56.316495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e0cc2-80b9-4478-b4f6-06924f324b5d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:56.366635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-284e0cc2-80b9-4478-b4f6-06924f324b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a9ef16092824511b11494d4388e3c58', 'request_id': 'req-284e0cc2-80b9-4478-b4f6-06924f324b5d'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:56.374509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-284e0cc2-80b9-4478-b4f6-06924f324b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a9ef16092824511b11494d4388e3c58', 'request_id': 'req-284e0cc2-80b9-4478-b4f6-06924f324b5d'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:56.382103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-284e0cc2-80b9-4478-b4f6-06924f324b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a9ef16092824511b11494d4388e3c58', 'request_id': 'req-284e0cc2-80b9-4478-b4f6-06924f324b5d'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:56.394476 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-284e0cc2-80b9-4478-b4f6-06924f324b5d admin admin] The token cache is being invalidate because project 2a9ef16092824511b11494d4388e3c58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:31:56.394767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-284e0cc2-80b9-4478-b4f6-06924f324b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:56.395424 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-284e0cc2-80b9-4478-b4f6-06924f324b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:56.396952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1043/2111] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:31:56 2022] DELETE /identity/v3/projects/2a9ef16092824511b11494d4388e3c58 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:31:57.999621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f5949-0df8-48ea-9a31-08359c4a93e5 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:58.001079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-510f5949-0df8-48ea-9a31-08359c4a93e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:58.014129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f5949-0df8-48ea-9a31-08359c4a93e5 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:58.082505 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:58.083157 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:58.083436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:58.083712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] PATH_INFO: `/v3/users/9a20e97b5b4f49ddbe0a2f51b49999a0` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:58.084874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a20e97b5b4f49ddbe0a2f51b49999a0)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:58.086815 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:58.156165 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a20e97b5b4f49ddbe0a2f51b49999a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.156878 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a20e97b5b4f49ddbe0a2f51b49999a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.166768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a20e97b5b4f49ddbe0a2f51b49999a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.175464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a20e97b5b4f49ddbe0a2f51b49999a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.190184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a20e97b5b4f49ddbe0a2f51b49999a0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.202958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:58.209161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1069/2112] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:31:57 2022] DELETE /identity/v3/users/9a20e97b5b4f49ddbe0a2f51b49999a0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:58.213877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8916248-1f5e-4125-a910-f54f805a3077 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:58.214508 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8916248-1f5e-4125-a910-f54f805a3077 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:58.214508 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8916248-1f5e-4125-a910-f54f805a3077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:58.236243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d8916248-1f5e-4125-a910-f54f805a3077 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:58.237477 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d8916248-1f5e-4125-a910-f54f805a3077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:58.249720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:58.302817 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1044/2113] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:31:58.312953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:58.315195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:58.376027 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf275a9-b9d6-465c-b8c6-905072dac9e9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:58.380962 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa59eac-af14-4353-864c-53925a94be07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:58.381570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa59eac-af14-4353-864c-53925a94be07 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:58.381848 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa59eac-af14-4353-864c-53925a94be07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:58.382118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa59eac-af14-4353-864c-53925a94be07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:58.392894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa59eac-af14-4353-864c-53925a94be07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:58.394372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa59eac-af14-4353-864c-53925a94be07 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:58.405803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1070/2114] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:31:58.685270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8916248-1f5e-4125-a910-f54f805a3077 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:58.687037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d8916248-1f5e-4125-a910-f54f805a3077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:58.701581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8916248-1f5e-4125-a910-f54f805a3077 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:58.707875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:58.708566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:58.708989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:58.709383 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] PATH_INFO: `/v3/projects/a647b9e2276048ecb6ec88d50f4e405f` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:58.710641 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=a647b9e2276048ecb6ec88d50f4e405f)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:58.712718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:58.770998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a647b9e2276048ecb6ec88d50f4e405f', 'request_id': 'req-92233c95-011a-4995-8a52-1944f72a32ff'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.782182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a647b9e2276048ecb6ec88d50f4e405f', 'request_id': 'req-92233c95-011a-4995-8a52-1944f72a32ff'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.791842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a647b9e2276048ecb6ec88d50f4e405f', 'request_id': 'req-92233c95-011a-4995-8a52-1944f72a32ff'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.805527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] The token cache is being invalidate because project a647b9e2276048ecb6ec88d50f4e405f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:31:58.805801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.806456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.808206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1045/2115] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:31:58 2022] DELETE /identity/v3/projects/a647b9e2276048ecb6ec88d50f4e405f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:58.812278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa59eac-af14-4353-864c-53925a94be07 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:58.822371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa59eac-af14-4353-864c-53925a94be07 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:58.886233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ef7cb-bc90-4175-b42e-44841f72b786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:58.886963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ef7cb-bc90-4175-b42e-44841f72b786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:58.887296 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ef7cb-bc90-4175-b42e-44841f72b786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:58.887607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ef7cb-bc90-4175-b42e-44841f72b786 admin admin] PATH_INFO: `/v3/users/7c2e23e2869b45c98f003dbc2c0527fc` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:58.888967 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ef7cb-bc90-4175-b42e-44841f72b786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2e23e2869b45c98f003dbc2c0527fc)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:58.891309 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ef7cb-bc90-4175-b42e-44841f72b786 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:58.940817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-446ef7cb-bc90-4175-b42e-44841f72b786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2e23e2869b45c98f003dbc2c0527fc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.941470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-446ef7cb-bc90-4175-b42e-44841f72b786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2e23e2869b45c98f003dbc2c0527fc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.948870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-446ef7cb-bc90-4175-b42e-44841f72b786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2e23e2869b45c98f003dbc2c0527fc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.957458 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-446ef7cb-bc90-4175-b42e-44841f72b786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2e23e2869b45c98f003dbc2c0527fc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.970958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-446ef7cb-bc90-4175-b42e-44841f72b786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2e23e2869b45c98f003dbc2c0527fc'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:58.976978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1071/2116] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:31:58 2022] DELETE /identity/v3/users/7c2e23e2869b45c98f003dbc2c0527fc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:59.269825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:59.271365 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:59.285479 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92233c95-011a-4995-8a52-1944f72a32ff admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:59.354088 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:59.354860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:59.355060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:59.355949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd admin admin] PATH_INFO: `/v3/projects/d8215d6ffeba47c89f17810d2b826d57` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:59.357145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8215d6ffeba47c89f17810d2b826d57)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:59.359085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:59.408093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:59.412277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8215d6ffeba47c89f17810d2b826d57', 'request_id': 'req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:59.421943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8215d6ffeba47c89f17810d2b826d57', 'request_id': 'req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:59.429881 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8215d6ffeba47c89f17810d2b826d57', 'request_id': 'req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:59.444633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd admin admin] The token cache is being invalidate because project d8215d6ffeba47c89f17810d2b826d57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:31:59.444923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:59.445584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2361af2-eb9e-4f7f-af1e-b0f44f0012fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:31:59.447109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1046/2117] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:31:59 2022] DELETE /identity/v3/projects/d8215d6ffeba47c89f17810d2b826d57 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:59.453814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461cad9-b408-4504-a475-e4bd9b149bc3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:59.454345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461cad9-b408-4504-a475-e4bd9b149bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:59.454990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461cad9-b408-4504-a475-e4bd9b149bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:59.476732 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c461cad9-b408-4504-a475-e4bd9b149bc3 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:59.477963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c461cad9-b408-4504-a475-e4bd9b149bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:59.497280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:31:59.555637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1072/2118] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:59.567557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d021de-b01b-4ec1-a2aa-01368fcfa493 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:59.567865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d021de-b01b-4ec1-a2aa-01368fcfa493 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:59.568186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d021de-b01b-4ec1-a2aa-01368fcfa493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:59.591850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-64d021de-b01b-4ec1-a2aa-01368fcfa493 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:59.594603 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-64d021de-b01b-4ec1-a2aa-01368fcfa493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:31:59.604381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1047/2119] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:31:59.608457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461cad9-b408-4504-a475-e4bd9b149bc3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:59.620199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461cad9-b408-4504-a475-e4bd9b149bc3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:59.625745 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0e588-d619-4e35-979e-1baf4a585541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:59.626971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0e588-d619-4e35-979e-1baf4a585541 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:59.628058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0e588-d619-4e35-979e-1baf4a585541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:59.628058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0e588-d619-4e35-979e-1baf4a585541 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:59.628605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0e588-d619-4e35-979e-1baf4a585541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:31:59.630122 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0e588-d619-4e35-979e-1baf4a585541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:59.631757 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa0e588-d619-4e35-979e-1baf4a585541 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:59.649189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1073/2120] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:31:59.650156 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d021de-b01b-4ec1-a2aa-01368fcfa493 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:59.662621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d021de-b01b-4ec1-a2aa-01368fcfa493 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:59.680189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd86cb2-fedb-4d2c-8603-ff9cbefdecf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:59.680837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd86cb2-fedb-4d2c-8603-ff9cbefdecf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:59.681105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd86cb2-fedb-4d2c-8603-ff9cbefdecf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:59.681377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd86cb2-fedb-4d2c-8603-ff9cbefdecf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:59.682032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd86cb2-fedb-4d2c-8603-ff9cbefdecf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:59.683987 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd86cb2-fedb-4d2c-8603-ff9cbefdecf6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:59.719658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1048/2121] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:31:59 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:31:59.721328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0e588-d619-4e35-979e-1baf4a585541 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:59.732860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0e588-d619-4e35-979e-1baf4a585541 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:59.795023 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a549435-db14-4271-b6b1-fbc057028c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:59.795753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a549435-db14-4271-b6b1-fbc057028c91 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:59.796066 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a549435-db14-4271-b6b1-fbc057028c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:59.796339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a549435-db14-4271-b6b1-fbc057028c91 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:59.797060 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a549435-db14-4271-b6b1-fbc057028c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:59.799382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a549435-db14-4271-b6b1-fbc057028c91 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:59.822287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1074/2122] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:31:59 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:31:59.827898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd86cb2-fedb-4d2c-8603-ff9cbefdecf6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:59.838914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd86cb2-fedb-4d2c-8603-ff9cbefdecf6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:59.843450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243e33c-063d-4bb0-848c-7a2c2097561e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:59.844061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243e33c-063d-4bb0-848c-7a2c2097561e admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:59.844340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243e33c-063d-4bb0-848c-7a2c2097561e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:59.844606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243e33c-063d-4bb0-848c-7a2c2097561e admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:59.845851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243e33c-063d-4bb0-848c-7a2c2097561e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:59.847445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243e33c-063d-4bb0-848c-7a2c2097561e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:59.860120 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1049/2123] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:59.864180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a549435-db14-4271-b6b1-fbc057028c91 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:59.874298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a549435-db14-4271-b6b1-fbc057028c91 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:59.878911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429e417-1606-4594-b2af-cb31d613dabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:59.879511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429e417-1606-4594-b2af-cb31d613dabb admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:59.879794 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429e417-1606-4594-b2af-cb31d613dabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:59.880101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429e417-1606-4594-b2af-cb31d613dabb admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:59.880743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429e417-1606-4594-b2af-cb31d613dabb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:59.882334 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429e417-1606-4594-b2af-cb31d613dabb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:59.895191 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1075/2124] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:31:59.899364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243e33c-063d-4bb0-848c-7a2c2097561e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:31:59.909637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243e33c-063d-4bb0-848c-7a2c2097561e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:31:59.914846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eac655-48d3-4426-a47a-1da53c2c163f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:31:59.915532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eac655-48d3-4426-a47a-1da53c2c163f admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:59.916387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eac655-48d3-4426-a47a-1da53c2c163f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:59.916387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eac655-48d3-4426-a47a-1da53c2c163f admin admin] PATH_INFO: `/v3/projects/26858eb3402b4eb7a6b9bddc75247fe0/users/c1c55ea57b6f4a99b20350f0f463ee35/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:59.933341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eac655-48d3-4426-a47a-1da53c2c163f admin admin] RBAC: Authorizing `identity:create_grant(project_id=26858eb3402b4eb7a6b9bddc75247fe0, user_id=c1c55ea57b6f4a99b20350f0f463ee35, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:31:59.936105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eac655-48d3-4426-a47a-1da53c2c163f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:31:59.954130 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1050/2125] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:31:59 2022] PUT /identity/v3/projects/26858eb3402b4eb7a6b9bddc75247fe0/users/c1c55ea57b6f4a99b20350f0f463ee35/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:31:59.959937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767b450-d2e0-4174-9175-ef831145b28e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:31:59.960209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767b450-d2e0-4174-9175-ef831145b28e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:31:59.960580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767b450-d2e0-4174-9175-ef831145b28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:31:59.980441 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9767b450-d2e0-4174-9175-ef831145b28e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:31:59.981679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9767b450-d2e0-4174-9175-ef831145b28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:00.043094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1076/2126] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:32:00.053047 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eac655-48d3-4426-a47a-1da53c2c163f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:00.055239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c8eac655-48d3-4426-a47a-1da53c2c163f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:00.114765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eac655-48d3-4426-a47a-1da53c2c163f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:00.118421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65a779-61f4-446e-9995-68c1d88cbedd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:00.119020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65a779-61f4-446e-9995-68c1d88cbedd service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:00.119394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65a779-61f4-446e-9995-68c1d88cbedd service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:00.119743 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65a779-61f4-446e-9995-68c1d88cbedd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:00.131007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab65a779-61f4-446e-9995-68c1d88cbedd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:00.132494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab65a779-61f4-446e-9995-68c1d88cbedd service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:00.143153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1051/2127] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:32:00.874036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc24c3e-c89c-4ec3-b9ca-45e912a45e9b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:00.874503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc24c3e-c89c-4ec3-b9ca-45e912a45e9b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:00.874604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc24c3e-c89c-4ec3-b9ca-45e912a45e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:00.913775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-dbc24c3e-c89c-4ec3-b9ca-45e912a45e9b None None] MFA Rules not processed for user `c1c55ea57b6f4a99b20350f0f463ee35`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:00.915074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dbc24c3e-c89c-4ec3-b9ca-45e912a45e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:00.937197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:01.031581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1077/2128] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:01.036486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65a779-61f4-446e-9995-68c1d88cbedd service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:01.049861 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65a779-61f4-446e-9995-68c1d88cbedd service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:01.060285 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be179c-a15c-48a8-b913-7499dc1742cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:01.061186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be179c-a15c-48a8-b913-7499dc1742cb admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:01.061644 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be179c-a15c-48a8-b913-7499dc1742cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:01.062077 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be179c-a15c-48a8-b913-7499dc1742cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:01.062884 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be179c-a15c-48a8-b913-7499dc1742cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:01.066161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be179c-a15c-48a8-b913-7499dc1742cb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:01.106813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1052/2129] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:32:01 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:32:01.111162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc24c3e-c89c-4ec3-b9ca-45e912a45e9b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:01.129973 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc24c3e-c89c-4ec3-b9ca-45e912a45e9b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:01.217096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79ff43-62f8-4346-aa58-d6af527cd78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:01.217532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79ff43-62f8-4346-aa58-d6af527cd78e admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:01.217930 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79ff43-62f8-4346-aa58-d6af527cd78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:01.218066 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79ff43-62f8-4346-aa58-d6af527cd78e admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:01.218704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79ff43-62f8-4346-aa58-d6af527cd78e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:01.221415 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79ff43-62f8-4346-aa58-d6af527cd78e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:01.259211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1078/2130] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:32:01 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:01.263593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be179c-a15c-48a8-b913-7499dc1742cb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:01.278910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be179c-a15c-48a8-b913-7499dc1742cb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:01.288065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c92b84-64e7-4acd-854c-d8542cda295d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:01.288820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c92b84-64e7-4acd-854c-d8542cda295d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:01.289203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c92b84-64e7-4acd-854c-d8542cda295d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:01.290595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c92b84-64e7-4acd-854c-d8542cda295d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:01.291432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c92b84-64e7-4acd-854c-d8542cda295d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:01.293724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c92b84-64e7-4acd-854c-d8542cda295d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:01.314443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1053/2131] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:32:01.319603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79ff43-62f8-4346-aa58-d6af527cd78e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:01.330669 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79ff43-62f8-4346-aa58-d6af527cd78e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:01.336036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fd88f-deab-4471-8886-4e4b98e4ebda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:01.336713 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fd88f-deab-4471-8886-4e4b98e4ebda admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:01.337099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fd88f-deab-4471-8886-4e4b98e4ebda admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:01.337461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fd88f-deab-4471-8886-4e4b98e4ebda admin admin] PATH_INFO: `/v3/projects/5bc0ffc04bfa4bdd8e1f6be1dc9b719a/users/97ed3dc84acd4552b1d90e93e3284f12/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:01.355527 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fd88f-deab-4471-8886-4e4b98e4ebda admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc0ffc04bfa4bdd8e1f6be1dc9b719a, user_id=97ed3dc84acd4552b1d90e93e3284f12, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:01.358366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fd88f-deab-4471-8886-4e4b98e4ebda admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:01.374929 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1079/2132] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:32:01 2022] PUT /identity/v3/projects/5bc0ffc04bfa4bdd8e1f6be1dc9b719a/users/97ed3dc84acd4552b1d90e93e3284f12/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:01.379568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c92b84-64e7-4acd-854c-d8542cda295d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:01.394610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c92b84-64e7-4acd-854c-d8542cda295d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:01.443613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:01.480558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d9aa9-040d-40a4-bd20-9209448bf8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:01.481279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d9aa9-040d-40a4-bd20-9209448bf8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:01.481888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d9aa9-040d-40a4-bd20-9209448bf8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:01.481998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d9aa9-040d-40a4-bd20-9209448bf8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:01.482780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d9aa9-040d-40a4-bd20-9209448bf8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:01.484680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d9aa9-040d-40a4-bd20-9209448bf8c2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:01.502661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1054/2133] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:32:01.503447 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fd88f-deab-4471-8886-4e4b98e4ebda admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:01.514892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fd88f-deab-4471-8886-4e4b98e4ebda admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:01.521409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73d1b5-7d73-4f60-ad8a-4dd7482cba24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:01.522079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73d1b5-7d73-4f60-ad8a-4dd7482cba24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:01.522360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73d1b5-7d73-4f60-ad8a-4dd7482cba24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:01.522635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73d1b5-7d73-4f60-ad8a-4dd7482cba24 admin admin] PATH_INFO: `/v3/projects/5bc0ffc04bfa4bdd8e1f6be1dc9b719a/users/97ed3dc84acd4552b1d90e93e3284f12/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:01.524588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73d1b5-7d73-4f60-ad8a-4dd7482cba24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc0ffc04bfa4bdd8e1f6be1dc9b719a, user_id=97ed3dc84acd4552b1d90e93e3284f12, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:01.527275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73d1b5-7d73-4f60-ad8a-4dd7482cba24 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:01.538947 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1080/2134] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:32:01 2022] PUT /identity/v3/projects/5bc0ffc04bfa4bdd8e1f6be1dc9b719a/users/97ed3dc84acd4552b1d90e93e3284f12/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:32:02.382073 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f4232-b90b-4803-a3c8-99f33db81afb None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:02.382512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f4232-b90b-4803-a3c8-99f33db81afb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:02.382512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f4232-b90b-4803-a3c8-99f33db81afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:02.426873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-782f4232-b90b-4803-a3c8-99f33db81afb None None] MFA Rules not processed for user `97ed3dc84acd4552b1d90e93e3284f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:02.428764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-782f4232-b90b-4803-a3c8-99f33db81afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:02.591419 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1055/2135] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:02.605761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73d1b5-7d73-4f60-ad8a-4dd7482cba24 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:02.608185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2e73d1b5-7d73-4f60-ad8a-4dd7482cba24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:02.747984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73d1b5-7d73-4f60-ad8a-4dd7482cba24 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:02.752167 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b2d849-af02-49df-9641-8dcf1b3902e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:02.752911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2d849-af02-49df-9641-8dcf1b3902e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:02.753905 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2d849-af02-49df-9641-8dcf1b3902e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:02.754341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2d849-af02-49df-9641-8dcf1b3902e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:02.792850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:02.844965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b2d849-af02-49df-9641-8dcf1b3902e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:02.845953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b2d849-af02-49df-9641-8dcf1b3902e0 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:02.857276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1081/2136] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:32:04.321388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f4232-b90b-4803-a3c8-99f33db81afb None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:04.322834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-782f4232-b90b-4803-a3c8-99f33db81afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:04.334543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f4232-b90b-4803-a3c8-99f33db81afb None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:04.338086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acc15f-9919-46e3-90b3-5860ed56870f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:04.338678 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acc15f-9919-46e3-90b3-5860ed56870f service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:04.338960 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acc15f-9919-46e3-90b3-5860ed56870f service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:04.339233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acc15f-9919-46e3-90b3-5860ed56870f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:04.348304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acc15f-9919-46e3-90b3-5860ed56870f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:04.349736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acc15f-9919-46e3-90b3-5860ed56870f service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:04.359130 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1056/2137] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:32:06.982961 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b2d849-af02-49df-9641-8dcf1b3902e0 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:06.984810 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-08b2d849-af02-49df-9641-8dcf1b3902e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:07.083461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b2d849-af02-49df-9641-8dcf1b3902e0 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:07.102605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:07.103261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:07.103541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:07.103813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] PATH_INFO: `/v3/users/9902122301ed4431a3ec1276a7af71b2` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:07.104986 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9902122301ed4431a3ec1276a7af71b2)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:07.107239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:07.189346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9902122301ed4431a3ec1276a7af71b2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:07.193290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9902122301ed4431a3ec1276a7af71b2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:07.209804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9902122301ed4431a3ec1276a7af71b2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:07.221907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9902122301ed4431a3ec1276a7af71b2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:07.264667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9902122301ed4431a3ec1276a7af71b2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:07.274994 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1082/2138] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:32:06 2022] DELETE /identity/v3/users/9902122301ed4431a3ec1276a7af71b2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:07.280630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039b2e0-b919-45db-98c7-6a11a5cee84b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:07.281459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039b2e0-b919-45db-98c7-6a11a5cee84b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:07.281459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039b2e0-b919-45db-98c7-6a11a5cee84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:07.319216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b039b2e0-b919-45db-98c7-6a11a5cee84b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:07.323549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b039b2e0-b919-45db-98c7-6a11a5cee84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:07.424365 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1057/2139] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:07.443532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:07.445415 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:07.520211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:07.541279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb82a4-2136-4808-96b1-ea74fc7c65cc admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:07.545097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70920b23-b1e8-4189-b2f9-b0cc9465481a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:07.545703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70920b23-b1e8-4189-b2f9-b0cc9465481a service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:07.545983 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70920b23-b1e8-4189-b2f9-b0cc9465481a service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:07.546251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70920b23-b1e8-4189-b2f9-b0cc9465481a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:07.557378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70920b23-b1e8-4189-b2f9-b0cc9465481a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:07.559126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70920b23-b1e8-4189-b2f9-b0cc9465481a service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:07.570855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1083/2140] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:32:07.841520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039b2e0-b919-45db-98c7-6a11a5cee84b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:07.842947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b039b2e0-b919-45db-98c7-6a11a5cee84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:07.859757 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039b2e0-b919-45db-98c7-6a11a5cee84b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:07.865038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731392e4-da66-408c-b476-592be21774c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:07.865726 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731392e4-da66-408c-b476-592be21774c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:07.866012 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731392e4-da66-408c-b476-592be21774c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:07.866284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731392e4-da66-408c-b476-592be21774c3 admin admin] PATH_INFO: `/v3/projects/ebd316863fbf4a548fd793b741599881` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:07.867403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731392e4-da66-408c-b476-592be21774c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd316863fbf4a548fd793b741599881)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:07.870051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731392e4-da66-408c-b476-592be21774c3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:07.914989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:07.922364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-731392e4-da66-408c-b476-592be21774c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd316863fbf4a548fd793b741599881', 'request_id': 'req-731392e4-da66-408c-b476-592be21774c3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:07.929966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-731392e4-da66-408c-b476-592be21774c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd316863fbf4a548fd793b741599881', 'request_id': 'req-731392e4-da66-408c-b476-592be21774c3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:07.936348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-731392e4-da66-408c-b476-592be21774c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd316863fbf4a548fd793b741599881', 'request_id': 'req-731392e4-da66-408c-b476-592be21774c3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:07.950706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-731392e4-da66-408c-b476-592be21774c3 admin admin] The token cache is being invalidate because project ebd316863fbf4a548fd793b741599881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:32:07.951228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-731392e4-da66-408c-b476-592be21774c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:07.952213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-731392e4-da66-408c-b476-592be21774c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:07.953938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1058/2141] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:32:07 2022] DELETE /identity/v3/projects/ebd316863fbf4a548fd793b741599881 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:07.960397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0b7c9-6589-40a8-86f5-faac78fb0760 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:07.960625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0b7c9-6589-40a8-86f5-faac78fb0760 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:07.960896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0b7c9-6589-40a8-86f5-faac78fb0760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:07.980776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8cf0b7c9-6589-40a8-86f5-faac78fb0760 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:07.981970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8cf0b7c9-6589-40a8-86f5-faac78fb0760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:08.041181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1084/2142] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:08.049059 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408057ab-4b5a-4597-baa7-8e231cd1fb03 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:08.049344 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408057ab-4b5a-4597-baa7-8e231cd1fb03 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:08.049620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408057ab-4b5a-4597-baa7-8e231cd1fb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:08.069572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-408057ab-4b5a-4597-baa7-8e231cd1fb03 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:08.070778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-408057ab-4b5a-4597-baa7-8e231cd1fb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:08.078594 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1059/2143] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:08.083269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0b7c9-6589-40a8-86f5-faac78fb0760 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:08.092966 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0b7c9-6589-40a8-86f5-faac78fb0760 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:08.098705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f0585-1811-4db6-81de-83683cafab4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:08.099408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f0585-1811-4db6-81de-83683cafab4e admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:08.099712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f0585-1811-4db6-81de-83683cafab4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:08.099985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f0585-1811-4db6-81de-83683cafab4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:08.100655 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f0585-1811-4db6-81de-83683cafab4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:32:08.100956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f0585-1811-4db6-81de-83683cafab4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:08.102593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f0585-1811-4db6-81de-83683cafab4e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:08.114055 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1085/2144] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:32:08.118314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408057ab-4b5a-4597-baa7-8e231cd1fb03 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:08.127756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408057ab-4b5a-4597-baa7-8e231cd1fb03 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:08.132566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d77d5-fc77-43d8-af36-61d3f9e2d222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:08.133153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d77d5-fc77-43d8-af36-61d3f9e2d222 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:08.133440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d77d5-fc77-43d8-af36-61d3f9e2d222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:08.133969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d77d5-fc77-43d8-af36-61d3f9e2d222 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:08.134600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d77d5-fc77-43d8-af36-61d3f9e2d222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:08.136540 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d77d5-fc77-43d8-af36-61d3f9e2d222 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:08.164384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1060/2145] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:32:08 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:08.168547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f0585-1811-4db6-81de-83683cafab4e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:08.178529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f0585-1811-4db6-81de-83683cafab4e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:08.237066 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966920ea-b532-41f5-a682-fc7efcfc3ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:08.237688 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966920ea-b532-41f5-a682-fc7efcfc3ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:08.238006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966920ea-b532-41f5-a682-fc7efcfc3ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:08.238350 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966920ea-b532-41f5-a682-fc7efcfc3ad0 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:08.239066 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966920ea-b532-41f5-a682-fc7efcfc3ad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:08.241238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966920ea-b532-41f5-a682-fc7efcfc3ad0 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:08.269480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1086/2146] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:32:08 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:08.270860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d77d5-fc77-43d8-af36-61d3f9e2d222 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:08.282504 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d77d5-fc77-43d8-af36-61d3f9e2d222 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:08.287427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0e9e0-5596-4398-a03d-1cff80163441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:08.288079 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0e9e0-5596-4398-a03d-1cff80163441 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:08.288363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0e9e0-5596-4398-a03d-1cff80163441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:08.288639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0e9e0-5596-4398-a03d-1cff80163441 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:08.289290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0e9e0-5596-4398-a03d-1cff80163441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:08.292028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0e9e0-5596-4398-a03d-1cff80163441 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:08.305341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1061/2147] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:32:08.307120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966920ea-b532-41f5-a682-fc7efcfc3ad0 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:08.318249 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966920ea-b532-41f5-a682-fc7efcfc3ad0 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:08.323370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60421c9f-bb36-4a59-9108-3cdf9363e5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:08.324022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60421c9f-bb36-4a59-9108-3cdf9363e5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:08.324447 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60421c9f-bb36-4a59-9108-3cdf9363e5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:08.324722 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60421c9f-bb36-4a59-9108-3cdf9363e5c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:08.325368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60421c9f-bb36-4a59-9108-3cdf9363e5c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:08.327004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60421c9f-bb36-4a59-9108-3cdf9363e5c9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:08.339860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1087/2148] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:32:08.344113 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0e9e0-5596-4398-a03d-1cff80163441 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:08.354727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0e9e0-5596-4398-a03d-1cff80163441 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:08.359636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c1bd7-a8bb-45a3-b625-ca2c0d8a0d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:08.360434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1bd7-a8bb-45a3-b625-ca2c0d8a0d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:08.360974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1bd7-a8bb-45a3-b625-ca2c0d8a0d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:08.361257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1bd7-a8bb-45a3-b625-ca2c0d8a0d7b admin admin] PATH_INFO: `/v3/projects/183fd2ebaf054cfebad0bafa207f9967/users/25cdba6a3bec49b4928b3fd953673060/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:08.376896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c1bd7-a8bb-45a3-b625-ca2c0d8a0d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=183fd2ebaf054cfebad0bafa207f9967, user_id=25cdba6a3bec49b4928b3fd953673060, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:08.379598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c1bd7-a8bb-45a3-b625-ca2c0d8a0d7b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:08.395551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1062/2149] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:32:08 2022] PUT /identity/v3/projects/183fd2ebaf054cfebad0bafa207f9967/users/25cdba6a3bec49b4928b3fd953673060/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:32:08.400791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434259f-682a-4515-bdaa-da650b124e8c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:08.401083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434259f-682a-4515-bdaa-da650b124e8c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:08.401355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3434259f-682a-4515-bdaa-da650b124e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:08.420601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3434259f-682a-4515-bdaa-da650b124e8c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:08.422031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3434259f-682a-4515-bdaa-da650b124e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:08.477018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:08.493103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1088/2150] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:08.505443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c1bd7-a8bb-45a3-b625-ca2c0d8a0d7b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:08.507354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-311c1bd7-a8bb-45a3-b625-ca2c0d8a0d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:08.580245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c1bd7-a8bb-45a3-b625-ca2c0d8a0d7b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:08.588251 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e0a4c-f0ab-4d34-9ee2-446196262afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:08.588956 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e0a4c-f0ab-4d34-9ee2-446196262afe service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:08.589245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e0a4c-f0ab-4d34-9ee2-446196262afe service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:08.589624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e0a4c-f0ab-4d34-9ee2-446196262afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:08.611796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e0a4c-f0ab-4d34-9ee2-446196262afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:08.613499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e0a4c-f0ab-4d34-9ee2-446196262afe service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:08.641503 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1063/2151] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:32:14.006018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8b280-bd50-4231-8621-b6a7531db367 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:14.006018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8b280-bd50-4231-8621-b6a7531db367 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:14.006662 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8b280-bd50-4231-8621-b6a7531db367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:14.044603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6ac8b280-bd50-4231-8621-b6a7531db367 None None] MFA Rules not processed for user `25cdba6a3bec49b4928b3fd953673060`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:14.045873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6ac8b280-bd50-4231-8621-b6a7531db367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:14.126265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1089/2152] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:14.130781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e0a4c-f0ab-4d34-9ee2-446196262afe service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:14.145316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e0a4c-f0ab-4d34-9ee2-446196262afe service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:14.154695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e9d46-3c09-4179-9595-3ff2653d45f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:14.155310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e9d46-3c09-4179-9595-3ff2653d45f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:14.155694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e9d46-3c09-4179-9595-3ff2653d45f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:14.155848 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e9d46-3c09-4179-9595-3ff2653d45f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:14.156531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e9d46-3c09-4179-9595-3ff2653d45f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:14.171408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e9d46-3c09-4179-9595-3ff2653d45f3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:14.227704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1064/2153] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:32:14 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:14.233340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8b280-bd50-4231-8621-b6a7531db367 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:14.254132 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8b280-bd50-4231-8621-b6a7531db367 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:14.371689 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166377f-f96a-408f-ad67-631e7972f486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:14.372590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166377f-f96a-408f-ad67-631e7972f486 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:14.373078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166377f-f96a-408f-ad67-631e7972f486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:14.374154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166377f-f96a-408f-ad67-631e7972f486 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:14.375308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166377f-f96a-408f-ad67-631e7972f486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:14.379301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166377f-f96a-408f-ad67-631e7972f486 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:14.455618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1090/2154] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:32:14 2022] POST /identity/v3/users => generated 296 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:14.459597 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e9d46-3c09-4179-9595-3ff2653d45f3 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:14.473952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e9d46-3c09-4179-9595-3ff2653d45f3 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:14.488177 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac67b4e-0303-4a71-b734-244a50f64120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:14.489720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac67b4e-0303-4a71-b734-244a50f64120 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:14.489997 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac67b4e-0303-4a71-b734-244a50f64120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:14.490408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac67b4e-0303-4a71-b734-244a50f64120 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:14.491815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac67b4e-0303-4a71-b734-244a50f64120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:14.494651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac67b4e-0303-4a71-b734-244a50f64120 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:14.541130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166377f-f96a-408f-ad67-631e7972f486 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:14.559163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1065/2155] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:32:14.569161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166377f-f96a-408f-ad67-631e7972f486 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:14.591476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d226b-70c6-4817-b179-8c9238c0280e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:14.592300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d226b-70c6-4817-b179-8c9238c0280e admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:14.592655 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d226b-70c6-4817-b179-8c9238c0280e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:14.593004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d226b-70c6-4817-b179-8c9238c0280e admin admin] PATH_INFO: `/v3/projects/d1072ad00e8d4793b264b6d4a8185c3b/users/a53200c02ebf45589ae1c2e080949363/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:14.629657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d226b-70c6-4817-b179-8c9238c0280e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1072ad00e8d4793b264b6d4a8185c3b, user_id=a53200c02ebf45589ae1c2e080949363, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:14.633393 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d226b-70c6-4817-b179-8c9238c0280e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:14.658300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1091/2156] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:32:14 2022] PUT /identity/v3/projects/d1072ad00e8d4793b264b6d4a8185c3b/users/a53200c02ebf45589ae1c2e080949363/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:14.659366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac67b4e-0303-4a71-b734-244a50f64120 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:14.669980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac67b4e-0303-4a71-b734-244a50f64120 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:14.731138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:14.785602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82442cd7-b277-4817-8bfe-78495e510071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:14.786137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82442cd7-b277-4817-8bfe-78495e510071 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:14.786637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82442cd7-b277-4817-8bfe-78495e510071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:14.786920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82442cd7-b277-4817-8bfe-78495e510071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:14.787718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82442cd7-b277-4817-8bfe-78495e510071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:14.789967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82442cd7-b277-4817-8bfe-78495e510071 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:14.802741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1066/2157] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:32:14.807813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d226b-70c6-4817-b179-8c9238c0280e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:14.819393 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d226b-70c6-4817-b179-8c9238c0280e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:14.826609 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300cea07-ca5c-4a35-a292-5970a7d65172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:14.827686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300cea07-ca5c-4a35-a292-5970a7d65172 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:14.827806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300cea07-ca5c-4a35-a292-5970a7d65172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:14.828149 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300cea07-ca5c-4a35-a292-5970a7d65172 admin admin] PATH_INFO: `/v3/projects/d1072ad00e8d4793b264b6d4a8185c3b/users/a53200c02ebf45589ae1c2e080949363/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:14.831000 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300cea07-ca5c-4a35-a292-5970a7d65172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1072ad00e8d4793b264b6d4a8185c3b, user_id=a53200c02ebf45589ae1c2e080949363, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:14.833823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300cea07-ca5c-4a35-a292-5970a7d65172 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:14.852398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1092/2158] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:32:14 2022] PUT /identity/v3/projects/d1072ad00e8d4793b264b6d4a8185c3b/users/a53200c02ebf45589ae1c2e080949363/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:16.246048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f14949-7fff-48b3-bf54-b1ba278da58b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:16.246538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f14949-7fff-48b3-bf54-b1ba278da58b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:16.246636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f14949-7fff-48b3-bf54-b1ba278da58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:16.280369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-86f14949-7fff-48b3-bf54-b1ba278da58b None None] MFA Rules not processed for user `f4c978e96d3248d59120c287ecc75e87`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:16.282579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-86f14949-7fff-48b3-bf54-b1ba278da58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:16.402142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1067/2159] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:19.940428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8c35e-3d06-45be-87a1-6c142d313f82 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:19.940936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8c35e-3d06-45be-87a1-6c142d313f82 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:19.941099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8c35e-3d06-45be-87a1-6c142d313f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:19.988950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-53a8c35e-3d06-45be-87a1-6c142d313f82 None None] MFA Rules not processed for user `a53200c02ebf45589ae1c2e080949363`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:19.990249 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-53a8c35e-3d06-45be-87a1-6c142d313f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:20.027154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:20.082823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1093/2160] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:20.097382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f14949-7fff-48b3-bf54-b1ba278da58b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:20.099145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-86f14949-7fff-48b3-bf54-b1ba278da58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:20.226105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f14949-7fff-48b3-bf54-b1ba278da58b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:20.239991 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32659cde-cf2d-4c95-8532-f3ba4f62c095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:20.240647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32659cde-cf2d-4c95-8532-f3ba4f62c095 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:20.240927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32659cde-cf2d-4c95-8532-f3ba4f62c095 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:20.241205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32659cde-cf2d-4c95-8532-f3ba4f62c095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:20.319455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32659cde-cf2d-4c95-8532-f3ba4f62c095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:20.321304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32659cde-cf2d-4c95-8532-f3ba4f62c095 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:20.331568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1068/2161] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:32:21.296581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f352-1827-47f5-9574-5a749bb36461 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:21.297107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f352-1827-47f5-9574-5a749bb36461 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:21.297107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018f352-1827-47f5-9574-5a749bb36461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:21.328260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0018f352-1827-47f5-9574-5a749bb36461 None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:21.330464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0018f352-1827-47f5-9574-5a749bb36461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:21.416525 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1094/2162] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 05:32:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:21.433006 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32659cde-cf2d-4c95-8532-f3ba4f62c095 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:21.434550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-32659cde-cf2d-4c95-8532-f3ba4f62c095 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:21.452174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:21.533208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32659cde-cf2d-4c95-8532-f3ba4f62c095 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:21.539082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55b4d6-290b-41a1-9bea-b92c8aad8ca5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:21.539274 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55b4d6-290b-41a1-9bea-b92c8aad8ca5 service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:21.539544 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55b4d6-290b-41a1-9bea-b92c8aad8ca5 service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:21.539816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55b4d6-290b-41a1-9bea-b92c8aad8ca5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:21.551219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55b4d6-290b-41a1-9bea-b92c8aad8ca5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:21.553416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55b4d6-290b-41a1-9bea-b92c8aad8ca5 service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:21.571247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1069/2163] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:32:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 05:32:22.304762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b1870-cc91-4d10-a6f6-73acbaa51ff0 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:22.305360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b1870-cc91-4d10-a6f6-73acbaa51ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:22.305500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b1870-cc91-4d10-a6f6-73acbaa51ff0 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:22.307224 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1095/2164] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 05:32:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:32:22.351116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cec89-fcfc-4cb8-ba4a-afbff6cb6def None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:22.351403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cec89-fcfc-4cb8-ba4a-afbff6cb6def None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:22.351682 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cec89-fcfc-4cb8-ba4a-afbff6cb6def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:22.396225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-fa8cec89-fcfc-4cb8-ba4a-afbff6cb6def None None] MFA Rules not processed for user `4e985df8ad6c4bc7b3cd1328924e3ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:22.397754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fa8cec89-fcfc-4cb8-ba4a-afbff6cb6def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:22.648911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1070/2165] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 05:32:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:32:22.691277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18638b3e-49b1-476b-b3d1-0abea41b1afc None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:22.691573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18638b3e-49b1-476b-b3d1-0abea41b1afc None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:22.691851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18638b3e-49b1-476b-b3d1-0abea41b1afc None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:22.693993 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1096/2166] 10.222.0.33 () {54 vars in 961 bytes} [Sun Sep 11 05:32:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:32:22.706039 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb5515-3b60-4a54-b558-dd44c68168d6 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:22.706411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb5515-3b60-4a54-b558-dd44c68168d6 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:22.706762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb5515-3b60-4a54-b558-dd44c68168d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:22.746626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-efeb5515-3b60-4a54-b558-dd44c68168d6 None None] MFA Rules not processed for user `4e985df8ad6c4bc7b3cd1328924e3ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:22.748150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-efeb5515-3b60-4a54-b558-dd44c68168d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:22.778841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1071/2167] 10.222.0.33 () {58 vars in 1058 bytes} [Sun Sep 11 05:32:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:22.788577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18638b3e-49b1-476b-b3d1-0abea41b1afc None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:22.822761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18638b3e-49b1-476b-b3d1-0abea41b1afc None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:22.834800 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce276d6-83d5-4b4b-bddb-13430bb0aed6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:22.835703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce276d6-83d5-4b4b-bddb-13430bb0aed6 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:22.836212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce276d6-83d5-4b4b-bddb-13430bb0aed6 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:22.836678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce276d6-83d5-4b4b-bddb-13430bb0aed6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:22.866750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce276d6-83d5-4b4b-bddb-13430bb0aed6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:22.868763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce276d6-83d5-4b4b-bddb-13430bb0aed6 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:22.888850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1097/2168] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:32:23.853674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb5515-3b60-4a54-b558-dd44c68168d6 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:23.855163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-efeb5515-3b60-4a54-b558-dd44c68168d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:23.954728 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb5515-3b60-4a54-b558-dd44c68168d6 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:23.961843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:23.962874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:23.963162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:23.963486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] PATH_INFO: `/v3/users/c1c55ea57b6f4a99b20350f0f463ee35` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:23.964895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c55ea57b6f4a99b20350f0f463ee35)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:23.967061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:24.054344 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:24.100040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c55ea57b6f4a99b20350f0f463ee35'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:24.110787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c55ea57b6f4a99b20350f0f463ee35'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:24.143951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c55ea57b6f4a99b20350f0f463ee35'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:24.144728 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c55ea57b6f4a99b20350f0f463ee35'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:24.145399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c55ea57b6f4a99b20350f0f463ee35'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:24.169859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1072/2169] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:32:23 2022] DELETE /identity/v3/users/c1c55ea57b6f4a99b20350f0f463ee35 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:24.475951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce276d6-83d5-4b4b-bddb-13430bb0aed6 service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:24.520054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce276d6-83d5-4b4b-bddb-13430bb0aed6 service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:24.675834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:24.677408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:24.677641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:24.677925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] PATH_INFO: `/v3/projects/26858eb3402b4eb7a6b9bddc75247fe0` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:24.679847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26858eb3402b4eb7a6b9bddc75247fe0)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:24.682437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:24.841041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26858eb3402b4eb7a6b9bddc75247fe0', 'request_id': 'req-26c6c4d6-19f1-49c0-8839-0d2d04288389'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:24.845931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:24.853520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26858eb3402b4eb7a6b9bddc75247fe0', 'request_id': 'req-26c6c4d6-19f1-49c0-8839-0d2d04288389'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:24.872718 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26858eb3402b4eb7a6b9bddc75247fe0', 'request_id': 'req-26c6c4d6-19f1-49c0-8839-0d2d04288389'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:24.895698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] The token cache is being invalidate because project 26858eb3402b4eb7a6b9bddc75247fe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:32:24.896024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:24.896977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:24.899810 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1098/2170] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:32:24 2022] DELETE /identity/v3/projects/26858eb3402b4eb7a6b9bddc75247fe0 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:24.905079 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:24.907611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:24.932615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767b023-3161-4381-87cf-92813ed25614 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:25.040686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bf0cd-30c3-42a5-8d4d-bbe0216119f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:25.041663 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bf0cd-30c3-42a5-8d4d-bbe0216119f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:25.042141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bf0cd-30c3-42a5-8d4d-bbe0216119f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:25.042580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bf0cd-30c3-42a5-8d4d-bbe0216119f3 admin admin] PATH_INFO: `/v3/users/97ed3dc84acd4552b1d90e93e3284f12` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:25.043952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bf0cd-30c3-42a5-8d4d-bbe0216119f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97ed3dc84acd4552b1d90e93e3284f12)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:25.046169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bf0cd-30c3-42a5-8d4d-bbe0216119f3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:25.118132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-fc2bf0cd-30c3-42a5-8d4d-bbe0216119f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97ed3dc84acd4552b1d90e93e3284f12'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:25.129696 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-fc2bf0cd-30c3-42a5-8d4d-bbe0216119f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97ed3dc84acd4552b1d90e93e3284f12'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:25.178624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-fc2bf0cd-30c3-42a5-8d4d-bbe0216119f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97ed3dc84acd4552b1d90e93e3284f12'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:25.179785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-fc2bf0cd-30c3-42a5-8d4d-bbe0216119f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97ed3dc84acd4552b1d90e93e3284f12'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:25.180663 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-fc2bf0cd-30c3-42a5-8d4d-bbe0216119f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97ed3dc84acd4552b1d90e93e3284f12'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:25.199602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1073/2171] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:32:24 2022] DELETE /identity/v3/users/97ed3dc84acd4552b1d90e93e3284f12 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:25.508054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:25.510985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:25.535246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6c4d6-19f1-49c0-8839-0d2d04288389 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:25.635109 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7845a-cdf4-45c2-ab79-b22f657b50b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:25.635685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7845a-cdf4-45c2-ab79-b22f657b50b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:25.635966 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7845a-cdf4-45c2-ab79-b22f657b50b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:25.636273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7845a-cdf4-45c2-ab79-b22f657b50b4 admin admin] PATH_INFO: `/v3/projects/5bc0ffc04bfa4bdd8e1f6be1dc9b719a` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:25.637993 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7845a-cdf4-45c2-ab79-b22f657b50b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc0ffc04bfa4bdd8e1f6be1dc9b719a)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:25.639357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7845a-cdf4-45c2-ab79-b22f657b50b4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:25.697176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aba7845a-cdf4-45c2-ab79-b22f657b50b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc0ffc04bfa4bdd8e1f6be1dc9b719a', 'request_id': 'req-aba7845a-cdf4-45c2-ab79-b22f657b50b4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:25.704610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aba7845a-cdf4-45c2-ab79-b22f657b50b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc0ffc04bfa4bdd8e1f6be1dc9b719a', 'request_id': 'req-aba7845a-cdf4-45c2-ab79-b22f657b50b4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:25.711170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aba7845a-cdf4-45c2-ab79-b22f657b50b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc0ffc04bfa4bdd8e1f6be1dc9b719a', 'request_id': 'req-aba7845a-cdf4-45c2-ab79-b22f657b50b4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:25.731382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aba7845a-cdf4-45c2-ab79-b22f657b50b4 admin admin] The token cache is being invalidate because project 5bc0ffc04bfa4bdd8e1f6be1dc9b719a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:32:25.731623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aba7845a-cdf4-45c2-ab79-b22f657b50b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:25.733114 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aba7845a-cdf4-45c2-ab79-b22f657b50b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:25.736528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1099/2172] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:32:25 2022] DELETE /identity/v3/projects/5bc0ffc04bfa4bdd8e1f6be1dc9b719a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:25.741998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365278d-13f4-4ce3-9eaa-ace432700bd5 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:25.742168 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365278d-13f4-4ce3-9eaa-ace432700bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:25.742535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365278d-13f4-4ce3-9eaa-ace432700bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:25.776037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7365278d-13f4-4ce3-9eaa-ace432700bd5 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:25.778486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7365278d-13f4-4ce3-9eaa-ace432700bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:25.820685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:25.876384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1074/2173] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:25.884748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da4bf8-6d24-46e7-a1c7-699adaaed074 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:25.885328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da4bf8-6d24-46e7-a1c7-699adaaed074 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:25.885846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da4bf8-6d24-46e7-a1c7-699adaaed074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:25.924990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a6da4bf8-6d24-46e7-a1c7-699adaaed074 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:25.926676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a6da4bf8-6d24-46e7-a1c7-699adaaed074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:25.940158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1100/2174] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:32:25.944791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365278d-13f4-4ce3-9eaa-ace432700bd5 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:25.959531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365278d-13f4-4ce3-9eaa-ace432700bd5 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:25.967040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f157e8-df32-49b6-9e49-5cddc1421c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:25.968083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f157e8-df32-49b6-9e49-5cddc1421c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:25.968940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f157e8-df32-49b6-9e49-5cddc1421c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:25.970758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f157e8-df32-49b6-9e49-5cddc1421c4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:25.972179 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f157e8-df32-49b6-9e49-5cddc1421c4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:32:25.972942 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f157e8-df32-49b6-9e49-5cddc1421c4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:25.975593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f157e8-df32-49b6-9e49-5cddc1421c4a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:26.012763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1075/2175] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 05:32:26.017762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da4bf8-6d24-46e7-a1c7-699adaaed074 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:26.037619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da4bf8-6d24-46e7-a1c7-699adaaed074 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:26.045644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cc9c0-10d8-461e-8c6f-ccb070ea902f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:26.049476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cc9c0-10d8-461e-8c6f-ccb070ea902f admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:26.050286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cc9c0-10d8-461e-8c6f-ccb070ea902f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:26.050833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cc9c0-10d8-461e-8c6f-ccb070ea902f admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:26.051927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cc9c0-10d8-461e-8c6f-ccb070ea902f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:26.056045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cc9c0-10d8-461e-8c6f-ccb070ea902f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:26.093589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1101/2176] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:32:26 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:32:26.097406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f157e8-df32-49b6-9e49-5cddc1421c4a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:26.109343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f157e8-df32-49b6-9e49-5cddc1421c4a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:26.209996 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68aa32f-ee25-464a-96df-bdda5e529c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:26.210498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68aa32f-ee25-464a-96df-bdda5e529c51 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:26.210856 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68aa32f-ee25-464a-96df-bdda5e529c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:26.211191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68aa32f-ee25-464a-96df-bdda5e529c51 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:26.211949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68aa32f-ee25-464a-96df-bdda5e529c51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:26.213995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68aa32f-ee25-464a-96df-bdda5e529c51 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:26.242656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1076/2177] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:32:26 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:26.248419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cc9c0-10d8-461e-8c6f-ccb070ea902f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:26.267618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cc9c0-10d8-461e-8c6f-ccb070ea902f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:26.277356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97bcf6-8eb2-4400-b664-58ab259ebb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:26.277982 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97bcf6-8eb2-4400-b664-58ab259ebb4f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:26.278269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97bcf6-8eb2-4400-b664-58ab259ebb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:26.278541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97bcf6-8eb2-4400-b664-58ab259ebb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:26.279189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97bcf6-8eb2-4400-b664-58ab259ebb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:26.287278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97bcf6-8eb2-4400-b664-58ab259ebb4f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:26.311785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1102/2178] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:32:26.316661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68aa32f-ee25-464a-96df-bdda5e529c51 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:26.331166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68aa32f-ee25-464a-96df-bdda5e529c51 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:26.343363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478fd62-f80b-4d34-81a8-ba82a8ec2ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:26.345026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478fd62-f80b-4d34-81a8-ba82a8ec2ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:26.345916 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478fd62-f80b-4d34-81a8-ba82a8ec2ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:26.346726 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478fd62-f80b-4d34-81a8-ba82a8ec2ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:26.348270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478fd62-f80b-4d34-81a8-ba82a8ec2ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:26.350957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478fd62-f80b-4d34-81a8-ba82a8ec2ec9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:26.370712 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1077/2179] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:32:26.372830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97bcf6-8eb2-4400-b664-58ab259ebb4f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:26.387726 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97bcf6-8eb2-4400-b664-58ab259ebb4f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:26.394989 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065bb80-d954-4272-b45c-94662e023124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:26.396032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065bb80-d954-4272-b45c-94662e023124 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:26.396345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065bb80-d954-4272-b45c-94662e023124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:26.396940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065bb80-d954-4272-b45c-94662e023124 admin admin] PATH_INFO: `/v3/projects/7345c04fd2b3437384aeb2d469966642/users/48bf0c9df73d4e70b14d9805a5d54fa2/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:26.435567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065bb80-d954-4272-b45c-94662e023124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7345c04fd2b3437384aeb2d469966642, user_id=48bf0c9df73d4e70b14d9805a5d54fa2, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:26.439360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065bb80-d954-4272-b45c-94662e023124 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:26.477525 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1103/2180] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:32:26 2022] PUT /identity/v3/projects/7345c04fd2b3437384aeb2d469966642/users/48bf0c9df73d4e70b14d9805a5d54fa2/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:32:26.481189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddeb68-98a8-46ab-9c9d-45a0e6ae2a4e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:26.481867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddeb68-98a8-46ab-9c9d-45a0e6ae2a4e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:26.482095 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddeb68-98a8-46ab-9c9d-45a0e6ae2a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:26.548348 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7cddeb68-98a8-46ab-9c9d-45a0e6ae2a4e None None] MFA Rules not processed for user `48bf0c9df73d4e70b14d9805a5d54fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:26.549866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7cddeb68-98a8-46ab-9c9d-45a0e6ae2a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:26.760898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1078/2181] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:26.765074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065bb80-d954-4272-b45c-94662e023124 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:26.827426 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:26.938916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065bb80-d954-4272-b45c-94662e023124 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:26.947263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41849282-a44f-4519-9c54-14bcf5fa97d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:26.948858 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41849282-a44f-4519-9c54-14bcf5fa97d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:26.949746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41849282-a44f-4519-9c54-14bcf5fa97d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:26.950546 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41849282-a44f-4519-9c54-14bcf5fa97d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:26.952212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41849282-a44f-4519-9c54-14bcf5fa97d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:26.956765 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41849282-a44f-4519-9c54-14bcf5fa97d9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:27.006402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1104/2182] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:32:26 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:27.007367 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddeb68-98a8-46ab-9c9d-45a0e6ae2a4e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:27.021516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddeb68-98a8-46ab-9c9d-45a0e6ae2a4e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:27.077176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:27.137626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864942-4539-496a-9a45-dabc1fd4d3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:27.138510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864942-4539-496a-9a45-dabc1fd4d3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:27.139234 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864942-4539-496a-9a45-dabc1fd4d3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:27.139234 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864942-4539-496a-9a45-dabc1fd4d3f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:27.139738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf864942-4539-496a-9a45-dabc1fd4d3f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:27.142991 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf864942-4539-496a-9a45-dabc1fd4d3f5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:27.178705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41849282-a44f-4519-9c54-14bcf5fa97d9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:27.184491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1079/2183] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:32:27 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:27.191708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41849282-a44f-4519-9c54-14bcf5fa97d9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:27.198284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a82a96-273b-4bf1-9acb-2deb9ef95c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:27.198951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a82a96-273b-4bf1-9acb-2deb9ef95c9f admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:27.199233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a82a96-273b-4bf1-9acb-2deb9ef95c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:27.199738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a82a96-273b-4bf1-9acb-2deb9ef95c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:27.200442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a82a96-273b-4bf1-9acb-2deb9ef95c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:27.202207 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a82a96-273b-4bf1-9acb-2deb9ef95c9f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:27.216231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1105/2184] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:32:27.221008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864942-4539-496a-9a45-dabc1fd4d3f5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:27.233128 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864942-4539-496a-9a45-dabc1fd4d3f5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:27.239090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72e65e-f8a7-45b7-aed6-e68391c4bc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:27.240407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72e65e-f8a7-45b7-aed6-e68391c4bc63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:27.240836 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72e65e-f8a7-45b7-aed6-e68391c4bc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:27.241332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72e65e-f8a7-45b7-aed6-e68391c4bc63 admin admin] PATH_INFO: `/v3/projects/a827bc860ec5485c80197de0fabb3c15/users/9d836941a33749089f32e0a6708580c3/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:27.268167 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72e65e-f8a7-45b7-aed6-e68391c4bc63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a827bc860ec5485c80197de0fabb3c15, user_id=9d836941a33749089f32e0a6708580c3, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:27.271681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72e65e-f8a7-45b7-aed6-e68391c4bc63 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:27.289716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1080/2185] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:32:27 2022] PUT /identity/v3/projects/a827bc860ec5485c80197de0fabb3c15/users/9d836941a33749089f32e0a6708580c3/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:27.293723 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a82a96-273b-4bf1-9acb-2deb9ef95c9f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:27.307111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a82a96-273b-4bf1-9acb-2deb9ef95c9f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:27.464834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef81908-63b6-4fb7-95bb-6978b46d3f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:27.465996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef81908-63b6-4fb7-95bb-6978b46d3f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:27.466522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef81908-63b6-4fb7-95bb-6978b46d3f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:27.467098 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef81908-63b6-4fb7-95bb-6978b46d3f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:27.472499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef81908-63b6-4fb7-95bb-6978b46d3f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:27.475565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef81908-63b6-4fb7-95bb-6978b46d3f1c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:27.500121 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1106/2186] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:32:27.502646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72e65e-f8a7-45b7-aed6-e68391c4bc63 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:27.515337 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72e65e-f8a7-45b7-aed6-e68391c4bc63 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:27.521408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbad55-ada0-4d46-978a-b3df4727880f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:27.522207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbad55-ada0-4d46-978a-b3df4727880f admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:27.522496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbad55-ada0-4d46-978a-b3df4727880f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:27.522782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbad55-ada0-4d46-978a-b3df4727880f admin admin] PATH_INFO: `/v3/projects/a827bc860ec5485c80197de0fabb3c15/users/9d836941a33749089f32e0a6708580c3/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:27.524811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbad55-ada0-4d46-978a-b3df4727880f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a827bc860ec5485c80197de0fabb3c15, user_id=9d836941a33749089f32e0a6708580c3, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:27.527521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbad55-ada0-4d46-978a-b3df4727880f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:27.539278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1081/2187] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:32:27 2022] PUT /identity/v3/projects/a827bc860ec5485c80197de0fabb3c15/users/9d836941a33749089f32e0a6708580c3/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:27.550471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595aea00-0111-4c22-9299-b4fc639adbab None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:27.550895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595aea00-0111-4c22-9299-b4fc639adbab None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:27.551008 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595aea00-0111-4c22-9299-b4fc639adbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:27.584418 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-595aea00-0111-4c22-9299-b4fc639adbab None None] MFA Rules not processed for user `9d836941a33749089f32e0a6708580c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:27.585943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-595aea00-0111-4c22-9299-b4fc639adbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:27.695798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1107/2188] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:32:27.706727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbad55-ada0-4d46-978a-b3df4727880f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:27.709756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-74bbad55-ada0-4d46-978a-b3df4727880f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:27.797938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbad55-ada0-4d46-978a-b3df4727880f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:27.807052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e832f0-6684-4e18-891f-a8b94927056a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:27.810287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e832f0-6684-4e18-891f-a8b94927056a service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:27.810601 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e832f0-6684-4e18-891f-a8b94927056a service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:27.810882 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e832f0-6684-4e18-891f-a8b94927056a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:27.929210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e832f0-6684-4e18-891f-a8b94927056a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:27.930948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e832f0-6684-4e18-891f-a8b94927056a service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:27.951403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1082/2189] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:32:36.038182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595aea00-0111-4c22-9299-b4fc639adbab None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:36.045269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:36.054384 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595aea00-0111-4c22-9299-b4fc639adbab None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:36.058998 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ffafb8-b09a-4dd5-aff8-d8b14112228a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:36.060132 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffafb8-b09a-4dd5-aff8-d8b14112228a service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:36.060654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffafb8-b09a-4dd5-aff8-d8b14112228a service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:36.061133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffafb8-b09a-4dd5-aff8-d8b14112228a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:36.074367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ffafb8-b09a-4dd5-aff8-d8b14112228a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:36.076321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ffafb8-b09a-4dd5-aff8-d8b14112228a service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:36.102860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1108/2190] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:32:37.944354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=80,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:41.420690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e832f0-6684-4e18-891f-a8b94927056a service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:41.503231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e832f0-6684-4e18-891f-a8b94927056a service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:41.509344 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:41.510102 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:41.510474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:41.510751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] PATH_INFO: `/v3/users/48bf0c9df73d4e70b14d9805a5d54fa2` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:41.511905 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48bf0c9df73d4e70b14d9805a5d54fa2)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:41.513887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:41.572835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48bf0c9df73d4e70b14d9805a5d54fa2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:41.580703 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48bf0c9df73d4e70b14d9805a5d54fa2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:41.594175 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48bf0c9df73d4e70b14d9805a5d54fa2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:41.595022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48bf0c9df73d4e70b14d9805a5d54fa2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:41.595800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48bf0c9df73d4e70b14d9805a5d54fa2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:41.621645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1083/2191] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:32:41 2022] DELETE /identity/v3/users/48bf0c9df73d4e70b14d9805a5d54fa2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:41.627117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d868f-2fee-48ab-8cbf-d3c03dbf2810 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:41.627646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d868f-2fee-48ab-8cbf-d3c03dbf2810 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:41.627646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d868f-2fee-48ab-8cbf-d3c03dbf2810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:41.647792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2f2d868f-2fee-48ab-8cbf-d3c03dbf2810 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:41.649049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2f2d868f-2fee-48ab-8cbf-d3c03dbf2810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:41.722927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1109/2192] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:32:41.735966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:41.738037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:41.815758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd8bb5-e133-4649-bbc6-8b2b57290dc3 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:41.819620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda84b4-55f9-4c73-a746-71d8d7d6b0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:41.820336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda84b4-55f9-4c73-a746-71d8d7d6b0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:41.820695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda84b4-55f9-4c73-a746-71d8d7d6b0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:41.821056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda84b4-55f9-4c73-a746-71d8d7d6b0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:41.830943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda84b4-55f9-4c73-a746-71d8d7d6b0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:41.832386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda84b4-55f9-4c73-a746-71d8d7d6b0d1 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:41.842697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1084/2193] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:32:42.172527 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d868f-2fee-48ab-8cbf-d3c03dbf2810 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:42.174085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2f2d868f-2fee-48ab-8cbf-d3c03dbf2810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:42.190567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d868f-2fee-48ab-8cbf-d3c03dbf2810 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:42.201608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:42.202352 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:42.202702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:42.204180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] PATH_INFO: `/v3/projects/7345c04fd2b3437384aeb2d469966642` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:42.206120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7345c04fd2b3437384aeb2d469966642)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:42.209660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:42.283471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7345c04fd2b3437384aeb2d469966642', 'request_id': 'req-139b8169-dd30-4a59-879e-781f94104305'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:42.290321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7345c04fd2b3437384aeb2d469966642', 'request_id': 'req-139b8169-dd30-4a59-879e-781f94104305'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:42.298382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7345c04fd2b3437384aeb2d469966642', 'request_id': 'req-139b8169-dd30-4a59-879e-781f94104305'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:42.311289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] The token cache is being invalidate because project 7345c04fd2b3437384aeb2d469966642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:32:42.311589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:42.312234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:42.314860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1110/2194] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:32:42 2022] DELETE /identity/v3/projects/7345c04fd2b3437384aeb2d469966642 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:42.318744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda84b4-55f9-4c73-a746-71d8d7d6b0d1 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:42.329088 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda84b4-55f9-4c73-a746-71d8d7d6b0d1 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:42.349738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:42.410899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0dcce-cbc9-44a5-95a8-e0d974b53a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:42.411584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0dcce-cbc9-44a5-95a8-e0d974b53a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:42.411904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0dcce-cbc9-44a5-95a8-e0d974b53a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:42.412279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0dcce-cbc9-44a5-95a8-e0d974b53a16 admin admin] PATH_INFO: `/v3/users/9d836941a33749089f32e0a6708580c3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:42.413509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0dcce-cbc9-44a5-95a8-e0d974b53a16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d836941a33749089f32e0a6708580c3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:42.415509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0dcce-cbc9-44a5-95a8-e0d974b53a16 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:42.502554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dfb0dcce-cbc9-44a5-95a8-e0d974b53a16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d836941a33749089f32e0a6708580c3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:42.514946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dfb0dcce-cbc9-44a5-95a8-e0d974b53a16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d836941a33749089f32e0a6708580c3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:42.530198 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dfb0dcce-cbc9-44a5-95a8-e0d974b53a16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d836941a33749089f32e0a6708580c3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:42.531027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dfb0dcce-cbc9-44a5-95a8-e0d974b53a16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d836941a33749089f32e0a6708580c3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:42.531990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dfb0dcce-cbc9-44a5-95a8-e0d974b53a16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d836941a33749089f32e0a6708580c3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:42.551559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1085/2195] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:32:42 2022] DELETE /identity/v3/users/9d836941a33749089f32e0a6708580c3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:42.904746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:42.906772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:42.970084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b8169-dd30-4a59-879e-781f94104305 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:43.061203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c6526-741e-4a30-9505-cc17ac3c996b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:43.062230 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c6526-741e-4a30-9505-cc17ac3c996b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:43.063089 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c6526-741e-4a30-9505-cc17ac3c996b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:43.064128 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c6526-741e-4a30-9505-cc17ac3c996b admin admin] PATH_INFO: `/v3/projects/a827bc860ec5485c80197de0fabb3c15` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:43.065362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c6526-741e-4a30-9505-cc17ac3c996b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a827bc860ec5485c80197de0fabb3c15)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:43.076078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c6526-741e-4a30-9505-cc17ac3c996b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:43.097754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:43.145353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ab9c6526-741e-4a30-9505-cc17ac3c996b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a827bc860ec5485c80197de0fabb3c15', 'request_id': 'req-ab9c6526-741e-4a30-9505-cc17ac3c996b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:43.167156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ab9c6526-741e-4a30-9505-cc17ac3c996b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a827bc860ec5485c80197de0fabb3c15', 'request_id': 'req-ab9c6526-741e-4a30-9505-cc17ac3c996b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:43.176476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ab9c6526-741e-4a30-9505-cc17ac3c996b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a827bc860ec5485c80197de0fabb3c15', 'request_id': 'req-ab9c6526-741e-4a30-9505-cc17ac3c996b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:43.191318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ab9c6526-741e-4a30-9505-cc17ac3c996b admin admin] The token cache is being invalidate because project a827bc860ec5485c80197de0fabb3c15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:32:43.191649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ab9c6526-741e-4a30-9505-cc17ac3c996b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:43.192307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ab9c6526-741e-4a30-9505-cc17ac3c996b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:43.193934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1111/2196] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:32:42 2022] DELETE /identity/v3/projects/a827bc860ec5485c80197de0fabb3c15 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:43.200867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcef328-05ed-4332-a17c-7f98c43d5915 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:43.201145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcef328-05ed-4332-a17c-7f98c43d5915 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:43.201463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcef328-05ed-4332-a17c-7f98c43d5915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:43.224239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-bdcef328-05ed-4332-a17c-7f98c43d5915 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:43.225676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bdcef328-05ed-4332-a17c-7f98c43d5915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:43.299892 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1086/2197] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:43.310475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a76c22-a3bd-44de-bed5-e2b260f5f655 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:43.310776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a76c22-a3bd-44de-bed5-e2b260f5f655 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:43.311024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a76c22-a3bd-44de-bed5-e2b260f5f655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:43.335090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-15a76c22-a3bd-44de-bed5-e2b260f5f655 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:43.336710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-15a76c22-a3bd-44de-bed5-e2b260f5f655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:43.352379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1112/2198] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:32:43.354562 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcef328-05ed-4332-a17c-7f98c43d5915 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:43.365659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcef328-05ed-4332-a17c-7f98c43d5915 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:43.371183 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213c7b8-2ede-4c2b-8e99-2938922e4240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:43.371823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213c7b8-2ede-4c2b-8e99-2938922e4240 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:43.372216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213c7b8-2ede-4c2b-8e99-2938922e4240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:43.372408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213c7b8-2ede-4c2b-8e99-2938922e4240 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:43.373052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213c7b8-2ede-4c2b-8e99-2938922e4240 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:32:43.373441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213c7b8-2ede-4c2b-8e99-2938922e4240 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:43.375207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213c7b8-2ede-4c2b-8e99-2938922e4240 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:43.394003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1087/2199] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:32:43.395253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a76c22-a3bd-44de-bed5-e2b260f5f655 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:43.408301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a76c22-a3bd-44de-bed5-e2b260f5f655 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:43.420045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab009d4-258c-4d30-afc9-a4de19bad7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:43.420781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab009d4-258c-4d30-afc9-a4de19bad7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:43.420998 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab009d4-258c-4d30-afc9-a4de19bad7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:43.421264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab009d4-258c-4d30-afc9-a4de19bad7e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:43.422114 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab009d4-258c-4d30-afc9-a4de19bad7e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:43.424255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab009d4-258c-4d30-afc9-a4de19bad7e4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:43.463158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1113/2200] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:32:43 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:43.467679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213c7b8-2ede-4c2b-8e99-2938922e4240 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:43.485069 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213c7b8-2ede-4c2b-8e99-2938922e4240 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:43.533197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:43.571862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94490b8-b8f8-49dd-9123-17ba1c9596ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:43.572577 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94490b8-b8f8-49dd-9123-17ba1c9596ec admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:43.572847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94490b8-b8f8-49dd-9123-17ba1c9596ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:43.573409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94490b8-b8f8-49dd-9123-17ba1c9596ec admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:43.573956 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94490b8-b8f8-49dd-9123-17ba1c9596ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:43.576043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94490b8-b8f8-49dd-9123-17ba1c9596ec admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:43.606174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1088/2201] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:32:43 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:43.607535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab009d4-258c-4d30-afc9-a4de19bad7e4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:43.621749 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab009d4-258c-4d30-afc9-a4de19bad7e4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:43.628089 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41675ec2-9dda-416b-a129-96c86a124170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:43.628685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41675ec2-9dda-416b-a129-96c86a124170 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:43.628983 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41675ec2-9dda-416b-a129-96c86a124170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:43.629443 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41675ec2-9dda-416b-a129-96c86a124170 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:43.630045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41675ec2-9dda-416b-a129-96c86a124170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:43.631702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41675ec2-9dda-416b-a129-96c86a124170 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:43.644691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1114/2202] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:32:43.649814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94490b8-b8f8-49dd-9123-17ba1c9596ec admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:43.670936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94490b8-b8f8-49dd-9123-17ba1c9596ec admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:43.679217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e0d3e-632e-48bf-9f9f-5536f8c2021d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:43.680242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e0d3e-632e-48bf-9f9f-5536f8c2021d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:43.680613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e0d3e-632e-48bf-9f9f-5536f8c2021d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:43.680814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e0d3e-632e-48bf-9f9f-5536f8c2021d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:43.681497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e0d3e-632e-48bf-9f9f-5536f8c2021d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:43.683203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e0d3e-632e-48bf-9f9f-5536f8c2021d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:43.699812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1089/2203] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:32:43.701177 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41675ec2-9dda-416b-a129-96c86a124170 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:43.711951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41675ec2-9dda-416b-a129-96c86a124170 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:43.716787 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2f4d2-66aa-4980-9ece-4840ec565da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:43.717628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2f4d2-66aa-4980-9ece-4840ec565da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:43.718127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2f4d2-66aa-4980-9ece-4840ec565da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:43.718320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2f4d2-66aa-4980-9ece-4840ec565da3 admin admin] PATH_INFO: `/v3/projects/7a3ad38fe4384875a6ec220509978975/users/99c20c774be348dcaa410dcc003c1f9e/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:43.734977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2f4d2-66aa-4980-9ece-4840ec565da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a3ad38fe4384875a6ec220509978975, user_id=99c20c774be348dcaa410dcc003c1f9e, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:43.738186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2f4d2-66aa-4980-9ece-4840ec565da3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:43.755960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1115/2204] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:32:43 2022] PUT /identity/v3/projects/7a3ad38fe4384875a6ec220509978975/users/99c20c774be348dcaa410dcc003c1f9e/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:43.771512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d622d0b-8fb8-4576-869d-7c7803a2c7f8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:43.771801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d622d0b-8fb8-4576-869d-7c7803a2c7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:43.772116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d622d0b-8fb8-4576-869d-7c7803a2c7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:43.809600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7d622d0b-8fb8-4576-869d-7c7803a2c7f8 None None] MFA Rules not processed for user `99c20c774be348dcaa410dcc003c1f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:43.810906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7d622d0b-8fb8-4576-869d-7c7803a2c7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:43.894543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1090/2205] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:43.908362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2f4d2-66aa-4980-9ece-4840ec565da3 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:43.911217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8ab2f4d2-66aa-4980-9ece-4840ec565da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:43.989517 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2f4d2-66aa-4980-9ece-4840ec565da3 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:43.993713 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88925b-0bba-40c8-8a06-2e970defb457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:43.994325 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88925b-0bba-40c8-8a06-2e970defb457 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:43.994618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88925b-0bba-40c8-8a06-2e970defb457 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:43.994903 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88925b-0bba-40c8-8a06-2e970defb457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:44.004189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88925b-0bba-40c8-8a06-2e970defb457 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:44.005742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88925b-0bba-40c8-8a06-2e970defb457 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:44.018746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1116/2206] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:32:47.725324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d622d0b-8fb8-4576-869d-7c7803a2c7f8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:47.803317 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d622d0b-8fb8-4576-869d-7c7803a2c7f8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:47.809990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:47.810731 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:47.811015 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:47.811292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] PATH_INFO: `/v3/users/99c20c774be348dcaa410dcc003c1f9e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:47.812675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99c20c774be348dcaa410dcc003c1f9e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:47.814666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:47.863229 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:47.872474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99c20c774be348dcaa410dcc003c1f9e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:47.884298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99c20c774be348dcaa410dcc003c1f9e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:47.900513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99c20c774be348dcaa410dcc003c1f9e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:47.901569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99c20c774be348dcaa410dcc003c1f9e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:47.902428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99c20c774be348dcaa410dcc003c1f9e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:47.919841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1091/2207] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:32:47 2022] DELETE /identity/v3/users/99c20c774be348dcaa410dcc003c1f9e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:47.924826 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fa33f-2be8-4ea8-b162-02001829e556 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:47.925375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fa33f-2be8-4ea8-b162-02001829e556 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:47.925728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fa33f-2be8-4ea8-b162-02001829e556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:47.948603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-cf1fa33f-2be8-4ea8-b162-02001829e556 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:47.949834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cf1fa33f-2be8-4ea8-b162-02001829e556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:47.997389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:48.022087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1117/2208] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:48.033057 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:48.034884 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:48.134359 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc36a3-cae0-44b9-adb2-975c614cb5d2 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:48.142843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570b4bd-2c5e-481c-8677-191eb25f4292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:48.143475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570b4bd-2c5e-481c-8677-191eb25f4292 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:48.143755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570b4bd-2c5e-481c-8677-191eb25f4292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:48.144055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570b4bd-2c5e-481c-8677-191eb25f4292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:48.161139 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570b4bd-2c5e-481c-8677-191eb25f4292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:48.162621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570b4bd-2c5e-481c-8677-191eb25f4292 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:48.181012 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1092/2209] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:32:48.555041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fa33f-2be8-4ea8-b162-02001829e556 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:48.556533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cf1fa33f-2be8-4ea8-b162-02001829e556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:48.570608 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fa33f-2be8-4ea8-b162-02001829e556 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:48.575355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701df2f-087a-4745-bfc9-97002a76e57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:48.576025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701df2f-087a-4745-bfc9-97002a76e57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:48.576320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701df2f-087a-4745-bfc9-97002a76e57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:48.576599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701df2f-087a-4745-bfc9-97002a76e57d admin admin] PATH_INFO: `/v3/projects/7a3ad38fe4384875a6ec220509978975` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:48.578283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701df2f-087a-4745-bfc9-97002a76e57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a3ad38fe4384875a6ec220509978975)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:48.580908 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701df2f-087a-4745-bfc9-97002a76e57d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:48.636091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a701df2f-087a-4745-bfc9-97002a76e57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a3ad38fe4384875a6ec220509978975', 'request_id': 'req-a701df2f-087a-4745-bfc9-97002a76e57d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:48.642886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a701df2f-087a-4745-bfc9-97002a76e57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a3ad38fe4384875a6ec220509978975', 'request_id': 'req-a701df2f-087a-4745-bfc9-97002a76e57d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:48.654756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a701df2f-087a-4745-bfc9-97002a76e57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a3ad38fe4384875a6ec220509978975', 'request_id': 'req-a701df2f-087a-4745-bfc9-97002a76e57d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:48.673959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a701df2f-087a-4745-bfc9-97002a76e57d admin admin] The token cache is being invalidate because project 7a3ad38fe4384875a6ec220509978975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:32:48.674366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a701df2f-087a-4745-bfc9-97002a76e57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:48.675274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a701df2f-087a-4745-bfc9-97002a76e57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:48.677997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1118/2210] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:32:48 2022] DELETE /identity/v3/projects/7a3ad38fe4384875a6ec220509978975 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:48.685658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820b55f-87b5-4f3b-ab8b-fe7e935b1148 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:48.686003 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820b55f-87b5-4f3b-ab8b-fe7e935b1148 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:48.686316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820b55f-87b5-4f3b-ab8b-fe7e935b1148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:48.708949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-2820b55f-87b5-4f3b-ab8b-fe7e935b1148 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:48.710153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2820b55f-87b5-4f3b-ab8b-fe7e935b1148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:48.782040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1093/2211] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:48.793631 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2be302-0c6a-4bfc-83e0-86dce5265d15 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:48.794674 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2be302-0c6a-4bfc-83e0-86dce5265d15 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:48.795228 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2be302-0c6a-4bfc-83e0-86dce5265d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:48.856263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3c2be302-0c6a-4bfc-83e0-86dce5265d15 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:48.856263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3c2be302-0c6a-4bfc-83e0-86dce5265d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:48.868236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1119/2212] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:48.872597 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2820b55f-87b5-4f3b-ab8b-fe7e935b1148 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:48.883683 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2820b55f-87b5-4f3b-ab8b-fe7e935b1148 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:48.889100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b8e07-701b-4c6c-9a67-64c06a562130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:48.889735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b8e07-701b-4c6c-9a67-64c06a562130 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:48.890013 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b8e07-701b-4c6c-9a67-64c06a562130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:48.890308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b8e07-701b-4c6c-9a67-64c06a562130 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:48.891534 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b8e07-701b-4c6c-9a67-64c06a562130 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:32:48.892724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b8e07-701b-4c6c-9a67-64c06a562130 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:48.894597 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b8e07-701b-4c6c-9a67-64c06a562130 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:48.912460 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1094/2213] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:32:48.916982 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2be302-0c6a-4bfc-83e0-86dce5265d15 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:48.943596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2be302-0c6a-4bfc-83e0-86dce5265d15 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:48.943596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b21ad9-19fd-4927-9df4-76df7e64767d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:48.943596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b21ad9-19fd-4927-9df4-76df7e64767d admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:48.943596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b21ad9-19fd-4927-9df4-76df7e64767d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:48.951559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b21ad9-19fd-4927-9df4-76df7e64767d admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:48.951559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b21ad9-19fd-4927-9df4-76df7e64767d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:48.951559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b21ad9-19fd-4927-9df4-76df7e64767d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:48.992040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1120/2214] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:32:48 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:48.993792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b8e07-701b-4c6c-9a67-64c06a562130 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:49.004943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b8e07-701b-4c6c-9a67-64c06a562130 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:49.057031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:49.079556 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d0c59-97c2-4f92-8e04-370b63bbe0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:49.080209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d0c59-97c2-4f92-8e04-370b63bbe0f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:49.080554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d0c59-97c2-4f92-8e04-370b63bbe0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:49.080828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d0c59-97c2-4f92-8e04-370b63bbe0f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:49.081514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d0c59-97c2-4f92-8e04-370b63bbe0f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:49.083551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d0c59-97c2-4f92-8e04-370b63bbe0f2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:49.111455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1095/2215] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:32:48 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:49.115663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b21ad9-19fd-4927-9df4-76df7e64767d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:49.131387 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b21ad9-19fd-4927-9df4-76df7e64767d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:49.138123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255a3c4-4f91-4ed6-add8-49ff1c7e5546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:49.139223 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255a3c4-4f91-4ed6-add8-49ff1c7e5546 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:49.139717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255a3c4-4f91-4ed6-add8-49ff1c7e5546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:49.140284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255a3c4-4f91-4ed6-add8-49ff1c7e5546 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:49.141123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255a3c4-4f91-4ed6-add8-49ff1c7e5546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:49.143093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255a3c4-4f91-4ed6-add8-49ff1c7e5546 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:49.159913 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1121/2216] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:32:49.160587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d0c59-97c2-4f92-8e04-370b63bbe0f2 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:49.173338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d0c59-97c2-4f92-8e04-370b63bbe0f2 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:49.178933 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae1359-a5cf-4c11-9cce-65404f535387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:49.179569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae1359-a5cf-4c11-9cce-65404f535387 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:49.182102 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae1359-a5cf-4c11-9cce-65404f535387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:49.182451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae1359-a5cf-4c11-9cce-65404f535387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:49.183033 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae1359-a5cf-4c11-9cce-65404f535387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:49.184886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae1359-a5cf-4c11-9cce-65404f535387 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:49.200051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1096/2217] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:32:49.202260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255a3c4-4f91-4ed6-add8-49ff1c7e5546 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:49.213509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255a3c4-4f91-4ed6-add8-49ff1c7e5546 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:49.219095 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304ba17-701f-41e5-9a4a-ae96fd3a4995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:49.219809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304ba17-701f-41e5-9a4a-ae96fd3a4995 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:49.220111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304ba17-701f-41e5-9a4a-ae96fd3a4995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:49.220394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304ba17-701f-41e5-9a4a-ae96fd3a4995 admin admin] PATH_INFO: `/v3/projects/bbab28318bf84c88b2f3a774defb870f/users/9793a6256764417eb3ff1a8386368801/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:49.237717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304ba17-701f-41e5-9a4a-ae96fd3a4995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbab28318bf84c88b2f3a774defb870f, user_id=9793a6256764417eb3ff1a8386368801, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:49.240641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304ba17-701f-41e5-9a4a-ae96fd3a4995 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:49.261755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1122/2218] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:32:49 2022] PUT /identity/v3/projects/bbab28318bf84c88b2f3a774defb870f/users/9793a6256764417eb3ff1a8386368801/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:49.267591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92774b0-85e3-405b-8d3f-184e63c2a8ea None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:49.267878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92774b0-85e3-405b-8d3f-184e63c2a8ea None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:49.268187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92774b0-85e3-405b-8d3f-184e63c2a8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:49.296277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d92774b0-85e3-405b-8d3f-184e63c2a8ea None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:49.298631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d92774b0-85e3-405b-8d3f-184e63c2a8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:49.365318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1097/2219] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:32:49.375573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304ba17-701f-41e5-9a4a-ae96fd3a4995 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:49.377860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9304ba17-701f-41e5-9a4a-ae96fd3a4995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:49.455481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304ba17-701f-41e5-9a4a-ae96fd3a4995 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:49.460627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583719b-3cda-46c5-81af-1aff4f754af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:49.461908 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583719b-3cda-46c5-81af-1aff4f754af9 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:49.462132 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583719b-3cda-46c5-81af-1aff4f754af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:49.462692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583719b-3cda-46c5-81af-1aff4f754af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:49.477317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583719b-3cda-46c5-81af-1aff4f754af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:49.479757 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583719b-3cda-46c5-81af-1aff4f754af9 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:49.498925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1123/2220] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:32:50.182829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1872b7-35ff-4260-8612-d75fe4662516 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:50.183318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1872b7-35ff-4260-8612-d75fe4662516 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:50.183735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1872b7-35ff-4260-8612-d75fe4662516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:50.214626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-3b1872b7-35ff-4260-8612-d75fe4662516 None None] MFA Rules not processed for user `9793a6256764417eb3ff1a8386368801`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:50.215901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3b1872b7-35ff-4260-8612-d75fe4662516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:50.297474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1098/2221] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:50.308561 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583719b-3cda-46c5-81af-1aff4f754af9 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:50.309997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9583719b-3cda-46c5-81af-1aff4f754af9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:50.320813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:50.372431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583719b-3cda-46c5-81af-1aff4f754af9 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:50.377178 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eecb64-00f8-48f4-8bb9-3b9bcc53df1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:50.377786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eecb64-00f8-48f4-8bb9-3b9bcc53df1e service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:50.378094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eecb64-00f8-48f4-8bb9-3b9bcc53df1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:50.378369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eecb64-00f8-48f4-8bb9-3b9bcc53df1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:50.386952 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eecb64-00f8-48f4-8bb9-3b9bcc53df1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:50.388412 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eecb64-00f8-48f4-8bb9-3b9bcc53df1e service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:50.402311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1124/2222] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:32:51.368763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ec917-80b2-4f15-b18c-035f12f05e7e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:51.369201 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ec917-80b2-4f15-b18c-035f12f05e7e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:51.369289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ec917-80b2-4f15-b18c-035f12f05e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:51.389083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-fa1ec917-80b2-4f15-b18c-035f12f05e7e None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:51.390260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fa1ec917-80b2-4f15-b18c-035f12f05e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:51.440493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:51.449055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1099/2223] 10.222.0.33 () {58 vars in 967 bytes} [Sun Sep 11 05:32:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:51.461464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eecb64-00f8-48f4-8bb9-3b9bcc53df1e service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:51.462880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-88eecb64-00f8-48f4-8bb9-3b9bcc53df1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:51.534296 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eecb64-00f8-48f4-8bb9-3b9bcc53df1e service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:51.539212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4a88c-f400-4d43-8f51-914c2eecc0de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:51.539811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4a88c-f400-4d43-8f51-914c2eecc0de service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:51.540117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4a88c-f400-4d43-8f51-914c2eecc0de service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:51.540389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4a88c-f400-4d43-8f51-914c2eecc0de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:51.548844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4a88c-f400-4d43-8f51-914c2eecc0de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:51.550412 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4a88c-f400-4d43-8f51-914c2eecc0de service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:51.558419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1125/2224] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:32:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 05:32:52.823743 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950b591-eee2-4d7d-956e-adbf1142ea7c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:52.824249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950b591-eee2-4d7d-956e-adbf1142ea7c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:52.824338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950b591-eee2-4d7d-956e-adbf1142ea7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:52.847792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0950b591-eee2-4d7d-956e-adbf1142ea7c None None] MFA Rules not processed for user `9793a6256764417eb3ff1a8386368801`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:52.849124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0950b591-eee2-4d7d-956e-adbf1142ea7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:52.856949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1100/2225] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:53.685511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4a88c-f400-4d43-8f51-914c2eecc0de service swift] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:53.687295 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-36a4a88c-f400-4d43-8f51-914c2eecc0de service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:53.709522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4a88c-f400-4d43-8f51-914c2eecc0de service swift] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:53.715957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:53.716683 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:53.716991 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:53.717263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] PATH_INFO: `/v3/users/8424e74e566d4bf3b8840b5d5c20aa1f` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:53.719227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8424e74e566d4bf3b8840b5d5c20aa1f)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:53.721626 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:53.779348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8424e74e566d4bf3b8840b5d5c20aa1f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:53.780076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8424e74e566d4bf3b8840b5d5c20aa1f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:53.789089 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8424e74e566d4bf3b8840b5d5c20aa1f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:53.804994 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8424e74e566d4bf3b8840b5d5c20aa1f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:53.823562 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8424e74e566d4bf3b8840b5d5c20aa1f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:53.831784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1126/2226] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:32:53 2022] DELETE /identity/v3/users/8424e74e566d4bf3b8840b5d5c20aa1f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:32:54.013853 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950b591-eee2-4d7d-956e-adbf1142ea7c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:54.028206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950b591-eee2-4d7d-956e-adbf1142ea7c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:54.087689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:54.088287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:54.088569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:54.088840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2 admin admin] PATH_INFO: `/v3/projects/16e859bc783a445f9b951dd2363fa0ce` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:54.089971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16e859bc783a445f9b951dd2363fa0ce)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:54.092228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:54.136503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:54.138135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:54.153119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16e859bc783a445f9b951dd2363fa0ce', 'request_id': 'req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:54.153627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:54.163658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16e859bc783a445f9b951dd2363fa0ce', 'request_id': 'req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:54.177618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16e859bc783a445f9b951dd2363fa0ce', 'request_id': 'req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:54.188636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2 admin admin] The token cache is being invalidate because project 16e859bc783a445f9b951dd2363fa0ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:32:54.188924 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:54.189585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-09d034e8-8a96-4cc6-af1e-d1e6f0307ba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:32:54.191283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1101/2227] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:32:54 2022] DELETE /identity/v3/projects/16e859bc783a445f9b951dd2363fa0ce => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:54.195914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931e276-d816-47b1-8577-997257abf97a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:54.196233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931e276-d816-47b1-8577-997257abf97a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:54.196513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931e276-d816-47b1-8577-997257abf97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:54.214091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1d692-d836-478a-9806-307077828ad4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:54.216104 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-9931e276-d816-47b1-8577-997257abf97a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:54.217321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9931e276-d816-47b1-8577-997257abf97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:54.277043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ac6d7-1500-48b4-8222-ae93455da168 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:54.277662 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ac6d7-1500-48b4-8222-ae93455da168 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:54.277946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ac6d7-1500-48b4-8222-ae93455da168 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:54.278228 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ac6d7-1500-48b4-8222-ae93455da168 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:54.280090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-043ac6d7-1500-48b4-8222-ae93455da168 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:54.283799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1102/2228] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:32:54.291249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884531e-1603-485f-9f28-29220d69f415 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:54.291571 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884531e-1603-485f-9f28-29220d69f415 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:54.291922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884531e-1603-485f-9f28-29220d69f415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:54.292278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ac6d7-1500-48b4-8222-ae93455da168 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:54.293868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ac6d7-1500-48b4-8222-ae93455da168 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:54.304761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1127/2229] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:32:54.313131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4884531e-1603-485f-9f28-29220d69f415 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:54.314371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4884531e-1603-485f-9f28-29220d69f415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:54.322770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1103/2230] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:32:54.326887 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ac6d7-1500-48b4-8222-ae93455da168 service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:54.337157 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ac6d7-1500-48b4-8222-ae93455da168 service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:54.341795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ced441-ce89-4468-8471-68c5fab33ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:54.342335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ced441-ce89-4468-8471-68c5fab33ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:54.342621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ced441-ce89-4468-8471-68c5fab33ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:54.342897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ced441-ce89-4468-8471-68c5fab33ba6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:54.343573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ced441-ce89-4468-8471-68c5fab33ba6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:32:54.343875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ced441-ce89-4468-8471-68c5fab33ba6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:54.345501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ced441-ce89-4468-8471-68c5fab33ba6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:54.356916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1128/2231] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:32:54.361673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884531e-1603-485f-9f28-29220d69f415 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:54.370940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884531e-1603-485f-9f28-29220d69f415 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:54.376500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c4487-e980-4c63-b194-70e4e086f2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:54.377098 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c4487-e980-4c63-b194-70e4e086f2db admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:54.377378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c4487-e980-4c63-b194-70e4e086f2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:54.377650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c4487-e980-4c63-b194-70e4e086f2db admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:54.378286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c4487-e980-4c63-b194-70e4e086f2db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:54.380218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c4487-e980-4c63-b194-70e4e086f2db admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:54.396898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:54.409931 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1104/2232] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:32:54 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:54.414531 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ced441-ce89-4468-8471-68c5fab33ba6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:54.428370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ced441-ce89-4468-8471-68c5fab33ba6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:54.495261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a481ff-5c37-4dcb-8581-0715aa01f9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:54.495875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a481ff-5c37-4dcb-8581-0715aa01f9ee admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:54.496406 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a481ff-5c37-4dcb-8581-0715aa01f9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:54.496688 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a481ff-5c37-4dcb-8581-0715aa01f9ee admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:54.497322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a481ff-5c37-4dcb-8581-0715aa01f9ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:54.499383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a481ff-5c37-4dcb-8581-0715aa01f9ee admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:54.521315 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1129/2233] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:32:54 2022] POST /identity/v3/users => generated 282 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:32:54.525478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c4487-e980-4c63-b194-70e4e086f2db admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:54.536393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c4487-e980-4c63-b194-70e4e086f2db admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:54.541117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4eed0c-e51b-4a7f-9d11-a220d99864ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:54.541779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4eed0c-e51b-4a7f-9d11-a220d99864ff admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:54.542060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4eed0c-e51b-4a7f-9d11-a220d99864ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:54.542331 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4eed0c-e51b-4a7f-9d11-a220d99864ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:54.542978 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4eed0c-e51b-4a7f-9d11-a220d99864ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:54.544635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4eed0c-e51b-4a7f-9d11-a220d99864ff admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:54.558461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1105/2234] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:32:54.560009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a481ff-5c37-4dcb-8581-0715aa01f9ee admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:54.570579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a481ff-5c37-4dcb-8581-0715aa01f9ee admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:54.575214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ede5b-fd27-44bb-aa2a-ab949f587eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:54.575809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ede5b-fd27-44bb-aa2a-ab949f587eca admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:54.576257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ede5b-fd27-44bb-aa2a-ab949f587eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:54.576503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ede5b-fd27-44bb-aa2a-ab949f587eca admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:54.578222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ede5b-fd27-44bb-aa2a-ab949f587eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:54.578914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ede5b-fd27-44bb-aa2a-ab949f587eca admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:54.589941 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1130/2235] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:32:54.594096 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4eed0c-e51b-4a7f-9d11-a220d99864ff admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:54.604502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4eed0c-e51b-4a7f-9d11-a220d99864ff admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:54.609640 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc665f-9c9a-4084-872e-de5fb83bb490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:54.610497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc665f-9c9a-4084-872e-de5fb83bb490 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:54.610770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc665f-9c9a-4084-872e-de5fb83bb490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:54.611034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc665f-9c9a-4084-872e-de5fb83bb490 admin admin] PATH_INFO: `/v3/projects/365bf1b7d5ef4f8ab3e7ec697a5140c8/users/988b8bae685d494c97ca6ef7ba9d6cc5/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:54.626248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc665f-9c9a-4084-872e-de5fb83bb490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365bf1b7d5ef4f8ab3e7ec697a5140c8, user_id=988b8bae685d494c97ca6ef7ba9d6cc5, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:54.628937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc665f-9c9a-4084-872e-de5fb83bb490 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:54.642435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1106/2236] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:32:54 2022] PUT /identity/v3/projects/365bf1b7d5ef4f8ab3e7ec697a5140c8/users/988b8bae685d494c97ca6ef7ba9d6cc5/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:32:54.652867 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76615ef0-f8f3-4b5e-b0d7-dcaa56e5c822 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:54.653162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76615ef0-f8f3-4b5e-b0d7-dcaa56e5c822 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:54.653442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76615ef0-f8f3-4b5e-b0d7-dcaa56e5c822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:54.675453 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:32:54.686851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-76615ef0-f8f3-4b5e-b0d7-dcaa56e5c822 None None] MFA Rules not processed for user `988b8bae685d494c97ca6ef7ba9d6cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:54.688446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-76615ef0-f8f3-4b5e-b0d7-dcaa56e5c822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:54.761052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1131/2237] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:32:54.771963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc665f-9c9a-4084-872e-de5fb83bb490 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:32:54.773689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-07fc665f-9c9a-4084-872e-de5fb83bb490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:54.838439 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc665f-9c9a-4084-872e-de5fb83bb490 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:32:54.842255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a37557-fd7d-4bb0-9239-c3866b5c68bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:32:54.842861 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a37557-fd7d-4bb0-9239-c3866b5c68bc service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:54.843143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a37557-fd7d-4bb0-9239-c3866b5c68bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:54.843458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a37557-fd7d-4bb0-9239-c3866b5c68bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:54.852131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a37557-fd7d-4bb0-9239-c3866b5c68bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:32:54.853570 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a37557-fd7d-4bb0-9239-c3866b5c68bc service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:32:54.863053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1107/2238] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:32:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:32:58.457630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae062d7-a058-4112-b6eb-f65ff6e17f23 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:32:58.458085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae062d7-a058-4112-b6eb-f65ff6e17f23 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:32:58.458185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae062d7-a058-4112-b6eb-f65ff6e17f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:32:58.479266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0ae062d7-a058-4112-b6eb-f65ff6e17f23 None None] MFA Rules not processed for user `25cdba6a3bec49b4928b3fd953673060`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:32:58.480519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0ae062d7-a058-4112-b6eb-f65ff6e17f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:32:58.538628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1132/2239] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:32:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:33:04.859249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:33:08.531627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=53 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:33:12.899475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574773d-3281-4535-a3fb-e2794484c920 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:33:12.899965 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574773d-3281-4535-a3fb-e2794484c920 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:33:12.900107 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574773d-3281-4535-a3fb-e2794484c920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:33:12.928092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b574773d-3281-4535-a3fb-e2794484c920 None None] MFA Rules not processed for user `54b72a430d9d4d43954eb56d703079b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:33:12.929433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b574773d-3281-4535-a3fb-e2794484c920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:33:13.002489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1108/2240] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:33:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:33:22.995055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=25 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:33:32.442047 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae062d7-a058-4112-b6eb-f65ff6e17f23 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:33:32.443567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0ae062d7-a058-4112-b6eb-f65ff6e17f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:33:32.501622 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae062d7-a058-4112-b6eb-f65ff6e17f23 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:33:32.505198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f45eeb-bba3-49a4-9df2-42a5faef68b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:33:32.505799 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f45eeb-bba3-49a4-9df2-42a5faef68b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:33:32.506077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f45eeb-bba3-49a4-9df2-42a5faef68b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:33:32.506346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f45eeb-bba3-49a4-9df2-42a5faef68b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:33:32.508163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-15f45eeb-bba3-49a4-9df2-42a5faef68b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:33:32.573011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f45eeb-bba3-49a4-9df2-42a5faef68b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:33:32.574463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f45eeb-bba3-49a4-9df2-42a5faef68b2 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:33:32.585113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1133/2241] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:33:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:33:40.438485 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f519d85-dbbe-4d5d-bbca-e2455191ad35 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:33:40.438999 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f519d85-dbbe-4d5d-bbca-e2455191ad35 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:33:40.439118 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f519d85-dbbe-4d5d-bbca-e2455191ad35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:33:40.461202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-1f519d85-dbbe-4d5d-bbca-e2455191ad35 None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:33:40.462406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1f519d85-dbbe-4d5d-bbca-e2455191ad35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:33:40.518451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1109/2242] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 05:33:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 05:33:40.531646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f45eeb-bba3-49a4-9df2-42a5faef68b2 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:33:40.533061 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-15f45eeb-bba3-49a4-9df2-42a5faef68b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:33:40.605564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f45eeb-bba3-49a4-9df2-42a5faef68b2 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:33:40.611357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f052b00-e4bd-46c6-9af4-43d369b46c02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:33:40.611966 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f052b00-e4bd-46c6-9af4-43d369b46c02 service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:33:40.612274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f052b00-e4bd-46c6-9af4-43d369b46c02 service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:33:40.612546 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f052b00-e4bd-46c6-9af4-43d369b46c02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:33:40.623189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f052b00-e4bd-46c6-9af4-43d369b46c02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:33:40.625076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f052b00-e4bd-46c6-9af4-43d369b46c02 service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:33:40.633038 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1134/2243] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:33:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 05:33:50.511629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=25 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:33:50.629920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=82 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:33:59.590554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f519d85-dbbe-4d5d-bbca-e2455191ad35 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:33:59.652969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f519d85-dbbe-4d5d-bbca-e2455191ad35 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:33:59.657823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:33:59.658438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:33:59.658713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:33:59.658977 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] PATH_INFO: `/v3/users/988b8bae685d494c97ca6ef7ba9d6cc5` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:33:59.660100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=988b8bae685d494c97ca6ef7ba9d6cc5)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:33:59.662151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:33:59.709275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '988b8bae685d494c97ca6ef7ba9d6cc5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:33:59.716360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '988b8bae685d494c97ca6ef7ba9d6cc5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:33:59.728142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '988b8bae685d494c97ca6ef7ba9d6cc5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:33:59.728808 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '988b8bae685d494c97ca6ef7ba9d6cc5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:33:59.729421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '988b8bae685d494c97ca6ef7ba9d6cc5'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:33:59.742762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1110/2244] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:33:59 2022] DELETE /identity/v3/users/988b8bae685d494c97ca6ef7ba9d6cc5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:33:59.747316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c798969-0f39-4075-95aa-a9106c8dcdcb None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:33:59.747827 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c798969-0f39-4075-95aa-a9106c8dcdcb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:33:59.747947 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c798969-0f39-4075-95aa-a9106c8dcdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:33:59.767331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3c798969-0f39-4075-95aa-a9106c8dcdcb None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:33:59.768567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3c798969-0f39-4075-95aa-a9106c8dcdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:33:59.826594 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1135/2245] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:33:59.836358 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:33:59.837798 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:33:59.895615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011ff87-fbc8-4d32-940e-1bc2f92170b9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:33:59.899339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186920e-9bcb-400e-8881-f692bd116a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:33:59.899935 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186920e-9bcb-400e-8881-f692bd116a6c service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:33:59.900245 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186920e-9bcb-400e-8881-f692bd116a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:33:59.900520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186920e-9bcb-400e-8881-f692bd116a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:33:59.909851 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186920e-9bcb-400e-8881-f692bd116a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:33:59.911273 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186920e-9bcb-400e-8881-f692bd116a6c service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:33:59.921321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1111/2246] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:34:00.183337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c798969-0f39-4075-95aa-a9106c8dcdcb None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:00.184877 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3c798969-0f39-4075-95aa-a9106c8dcdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:00.197698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c798969-0f39-4075-95aa-a9106c8dcdcb None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:00.203244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbc066-6b6e-4136-a072-91e2aa92b177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:00.203894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbc066-6b6e-4136-a072-91e2aa92b177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:00.204203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbc066-6b6e-4136-a072-91e2aa92b177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:00.204474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbc066-6b6e-4136-a072-91e2aa92b177 admin admin] PATH_INFO: `/v3/projects/365bf1b7d5ef4f8ab3e7ec697a5140c8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:00.205642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbc066-6b6e-4136-a072-91e2aa92b177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=365bf1b7d5ef4f8ab3e7ec697a5140c8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:00.207487 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbc066-6b6e-4136-a072-91e2aa92b177 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:00.254458 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-badbc066-6b6e-4136-a072-91e2aa92b177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365bf1b7d5ef4f8ab3e7ec697a5140c8', 'request_id': 'req-badbc066-6b6e-4136-a072-91e2aa92b177'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:34:00.260117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-badbc066-6b6e-4136-a072-91e2aa92b177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365bf1b7d5ef4f8ab3e7ec697a5140c8', 'request_id': 'req-badbc066-6b6e-4136-a072-91e2aa92b177'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:34:00.267067 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-badbc066-6b6e-4136-a072-91e2aa92b177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365bf1b7d5ef4f8ab3e7ec697a5140c8', 'request_id': 'req-badbc066-6b6e-4136-a072-91e2aa92b177'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:34:00.279298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-badbc066-6b6e-4136-a072-91e2aa92b177 admin admin] The token cache is being invalidate because project 365bf1b7d5ef4f8ab3e7ec697a5140c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:34:00.279585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-badbc066-6b6e-4136-a072-91e2aa92b177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:34:00.280955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-badbc066-6b6e-4136-a072-91e2aa92b177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:34:00.282428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1136/2247] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:34:00 2022] DELETE /identity/v3/projects/365bf1b7d5ef4f8ab3e7ec697a5140c8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:34:00.288331 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9be9e-284d-4584-8d87-8c93038f37bd None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:00.288650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9be9e-284d-4584-8d87-8c93038f37bd None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:00.288933 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9be9e-284d-4584-8d87-8c93038f37bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:00.308424 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c4b9be9e-284d-4584-8d87-8c93038f37bd None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:34:00.309627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c4b9be9e-284d-4584-8d87-8c93038f37bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:00.326264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:34:00.369405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1112/2248] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:34:00.377406 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811203d5-e5cc-48f0-a12c-89a33824d453 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:00.377696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811203d5-e5cc-48f0-a12c-89a33824d453 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:00.377974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811203d5-e5cc-48f0-a12c-89a33824d453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:00.398530 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-811203d5-e5cc-48f0-a12c-89a33824d453 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:34:00.399743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-811203d5-e5cc-48f0-a12c-89a33824d453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:00.408561 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1137/2249] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:34:00.414354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9be9e-284d-4584-8d87-8c93038f37bd None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:00.424989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9be9e-284d-4584-8d87-8c93038f37bd None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:00.429520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1633e7-404e-483d-972a-673c2319d391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:00.430108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1633e7-404e-483d-972a-673c2319d391 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:00.430384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1633e7-404e-483d-972a-673c2319d391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:00.430661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1633e7-404e-483d-972a-673c2319d391 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:00.431298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1633e7-404e-483d-972a-673c2319d391 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 05:34:00.431648 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1633e7-404e-483d-972a-673c2319d391 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:00.433642 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1633e7-404e-483d-972a-673c2319d391 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:00.445564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1113/2250] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 05:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 05:34:00.449951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811203d5-e5cc-48f0-a12c-89a33824d453 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:00.459901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811203d5-e5cc-48f0-a12c-89a33824d453 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:00.464518 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7397c3-82e2-4ef4-8a75-6a9dbcd95666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:00.465104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7397c3-82e2-4ef4-8a75-6a9dbcd95666 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:00.465382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7397c3-82e2-4ef4-8a75-6a9dbcd95666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:00.465649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7397c3-82e2-4ef4-8a75-6a9dbcd95666 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:00.466275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7397c3-82e2-4ef4-8a75-6a9dbcd95666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:00.469206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7397c3-82e2-4ef4-8a75-6a9dbcd95666 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:00.495532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1138/2251] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:34:00 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:34:00.499681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1633e7-404e-483d-972a-673c2319d391 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:00.509954 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1633e7-404e-483d-972a-673c2319d391 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:00.569212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903bacf-354b-4e5c-a829-702993b52ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:00.569831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903bacf-354b-4e5c-a829-702993b52ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:00.570117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903bacf-354b-4e5c-a829-702993b52ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:00.570487 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903bacf-354b-4e5c-a829-702993b52ef5 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:00.571014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903bacf-354b-4e5c-a829-702993b52ef5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:00.573027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903bacf-354b-4e5c-a829-702993b52ef5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:00.598752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1114/2252] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:34:00 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:34:00.599899 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7397c3-82e2-4ef4-8a75-6a9dbcd95666 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:00.610482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7397c3-82e2-4ef4-8a75-6a9dbcd95666 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:00.615481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cecdb7-9e89-447e-b12e-ceac5ae014fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:00.616122 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cecdb7-9e89-447e-b12e-ceac5ae014fa admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:00.616408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cecdb7-9e89-447e-b12e-ceac5ae014fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:00.616685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cecdb7-9e89-447e-b12e-ceac5ae014fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:00.617372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cecdb7-9e89-447e-b12e-ceac5ae014fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:00.618977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cecdb7-9e89-447e-b12e-ceac5ae014fa admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:00.629730 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1139/2253] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:34:00.633689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903bacf-354b-4e5c-a829-702993b52ef5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:00.644634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903bacf-354b-4e5c-a829-702993b52ef5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:00.649392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412fe73-b5f4-41d0-afcf-dc2bfd3d5292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:00.649996 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412fe73-b5f4-41d0-afcf-dc2bfd3d5292 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:00.650280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412fe73-b5f4-41d0-afcf-dc2bfd3d5292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:00.650558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412fe73-b5f4-41d0-afcf-dc2bfd3d5292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:00.651203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412fe73-b5f4-41d0-afcf-dc2bfd3d5292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:00.652854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412fe73-b5f4-41d0-afcf-dc2bfd3d5292 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:00.664537 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1115/2254] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:34:00.668795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cecdb7-9e89-447e-b12e-ceac5ae014fa admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:00.678958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cecdb7-9e89-447e-b12e-ceac5ae014fa admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:00.683859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04220bbf-7110-455b-80f4-b6461f6490e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:00.684546 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04220bbf-7110-455b-80f4-b6461f6490e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:00.684828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04220bbf-7110-455b-80f4-b6461f6490e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:00.685101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04220bbf-7110-455b-80f4-b6461f6490e9 admin admin] PATH_INFO: `/v3/projects/da4ccd5b2fa14cd188de7c86f31d8632/users/c9acda7a0d8c4834aace017083393cd0/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:00.700833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04220bbf-7110-455b-80f4-b6461f6490e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da4ccd5b2fa14cd188de7c86f31d8632, user_id=c9acda7a0d8c4834aace017083393cd0, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:00.704154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04220bbf-7110-455b-80f4-b6461f6490e9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:00.718249 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1140/2255] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:34:00 2022] PUT /identity/v3/projects/da4ccd5b2fa14cd188de7c86f31d8632/users/c9acda7a0d8c4834aace017083393cd0/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:34:00.723427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75874ce-b62e-404e-89d5-77e653754f9f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:00.723714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75874ce-b62e-404e-89d5-77e653754f9f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:00.723987 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75874ce-b62e-404e-89d5-77e653754f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:00.743450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f75874ce-b62e-404e-89d5-77e653754f9f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:34:00.744734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f75874ce-b62e-404e-89d5-77e653754f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:00.804673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1116/2256] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:34:00.815049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04220bbf-7110-455b-80f4-b6461f6490e9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:00.816844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-04220bbf-7110-455b-80f4-b6461f6490e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:00.839083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:34:00.874845 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04220bbf-7110-455b-80f4-b6461f6490e9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:00.878564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db47f67-d01a-415f-8dd0-75f30db06421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:00.879168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db47f67-d01a-415f-8dd0-75f30db06421 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:00.879449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db47f67-d01a-415f-8dd0-75f30db06421 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:00.879744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db47f67-d01a-415f-8dd0-75f30db06421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:00.888845 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db47f67-d01a-415f-8dd0-75f30db06421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:00.890822 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db47f67-d01a-415f-8dd0-75f30db06421 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:00.900335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1141/2257] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:34:04.975972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d8f2b-7202-4abf-b53b-68d6dd922bab None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:04.977081 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d8f2b-7202-4abf-b53b-68d6dd922bab None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:04.977489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d8f2b-7202-4abf-b53b-68d6dd922bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:05.015854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-892d8f2b-7202-4abf-b53b-68d6dd922bab None None] MFA Rules not processed for user `c9acda7a0d8c4834aace017083393cd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:34:05.017979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-892d8f2b-7202-4abf-b53b-68d6dd922bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:05.042885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:34:05.149580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1117/2258] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:34:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:34:05.151286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db47f67-d01a-415f-8dd0-75f30db06421 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:05.171176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db47f67-d01a-415f-8dd0-75f30db06421 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:05.182665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5e037-b5ca-49b3-bb6a-bb859c04e10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:05.183298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5e037-b5ca-49b3-bb6a-bb859c04e10a admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:05.183583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5e037-b5ca-49b3-bb6a-bb859c04e10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:05.183859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5e037-b5ca-49b3-bb6a-bb859c04e10a admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:05.184822 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5e037-b5ca-49b3-bb6a-bb859c04e10a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:05.186992 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5e037-b5ca-49b3-bb6a-bb859c04e10a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:05.215416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1142/2259] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 05:34:05 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 05:34:05.220200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d8f2b-7202-4abf-b53b-68d6dd922bab None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:05.231545 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d8f2b-7202-4abf-b53b-68d6dd922bab None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:05.304704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a840e7-b039-44ea-a7d1-3789d9f024cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:05.305428 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a840e7-b039-44ea-a7d1-3789d9f024cf admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:05.305842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a840e7-b039-44ea-a7d1-3789d9f024cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:05.306180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a840e7-b039-44ea-a7d1-3789d9f024cf admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:05.306825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a840e7-b039-44ea-a7d1-3789d9f024cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:05.308797 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a840e7-b039-44ea-a7d1-3789d9f024cf admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:05.336098 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1118/2260] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 05:34:05 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 05:34:05.340373 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5e037-b5ca-49b3-bb6a-bb859c04e10a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:05.351662 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5e037-b5ca-49b3-bb6a-bb859c04e10a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:05.356380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7740f-3ad9-4880-a442-ee021418d9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:05.356997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7740f-3ad9-4880-a442-ee021418d9ba admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:05.357277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7740f-3ad9-4880-a442-ee021418d9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:05.357548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7740f-3ad9-4880-a442-ee021418d9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:05.358678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7740f-3ad9-4880-a442-ee021418d9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:05.361006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7740f-3ad9-4880-a442-ee021418d9ba admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:05.376120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1143/2261] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 05:34:05.380386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a840e7-b039-44ea-a7d1-3789d9f024cf admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:05.392143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a840e7-b039-44ea-a7d1-3789d9f024cf admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:05.397389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cea03-8205-4fbd-ab0e-7f51ea01c3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:05.397991 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cea03-8205-4fbd-ab0e-7f51ea01c3f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:05.398297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cea03-8205-4fbd-ab0e-7f51ea01c3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:05.398735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cea03-8205-4fbd-ab0e-7f51ea01c3f9 admin admin] PATH_INFO: `/v3/projects/b19162951cf94b4a86dbf3abeec331b8/users/3e0d17c9b0e7488fbd1319658455e391/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:05.415929 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596cea03-8205-4fbd-ab0e-7f51ea01c3f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19162951cf94b4a86dbf3abeec331b8, user_id=3e0d17c9b0e7488fbd1319658455e391, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:05.418648 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596cea03-8205-4fbd-ab0e-7f51ea01c3f9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:05.434121 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1119/2262] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:34:05 2022] PUT /identity/v3/projects/b19162951cf94b4a86dbf3abeec331b8/users/3e0d17c9b0e7488fbd1319658455e391/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 05:34:05.438860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7740f-3ad9-4880-a442-ee021418d9ba admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:05.449359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7740f-3ad9-4880-a442-ee021418d9ba admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:05.513649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd20dc-d5fa-4bac-8344-aaac8005adb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:05.514881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd20dc-d5fa-4bac-8344-aaac8005adb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:05.515289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd20dc-d5fa-4bac-8344-aaac8005adb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:05.515637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd20dc-d5fa-4bac-8344-aaac8005adb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:05.516418 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd20dc-d5fa-4bac-8344-aaac8005adb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:05.518100 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd20dc-d5fa-4bac-8344-aaac8005adb1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:05.528972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1144/2263] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 05:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 05:34:05.534542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cea03-8205-4fbd-ab0e-7f51ea01c3f9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:05.545320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cea03-8205-4fbd-ab0e-7f51ea01c3f9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:05.550369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b50c33-9b6c-4de8-91db-d736fe956b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:05.551084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b50c33-9b6c-4de8-91db-d736fe956b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:05.551365 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b50c33-9b6c-4de8-91db-d736fe956b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:05.551637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b50c33-9b6c-4de8-91db-d736fe956b16 admin admin] PATH_INFO: `/v3/projects/b19162951cf94b4a86dbf3abeec331b8/users/3e0d17c9b0e7488fbd1319658455e391/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:05.553633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b50c33-9b6c-4de8-91db-d736fe956b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19162951cf94b4a86dbf3abeec331b8, user_id=3e0d17c9b0e7488fbd1319658455e391, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:05.556364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b50c33-9b6c-4de8-91db-d736fe956b16 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:05.567674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1120/2264] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 05:34:05 2022] PUT /identity/v3/projects/b19162951cf94b4a86dbf3abeec331b8/users/3e0d17c9b0e7488fbd1319658455e391/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:34:06.902738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb85a4-012d-4a62-bfc6-943af3e3fdd8 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:06.903273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb85a4-012d-4a62-bfc6-943af3e3fdd8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:06.903273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb85a4-012d-4a62-bfc6-943af3e3fdd8 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:06.907686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1145/2265] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 05:34:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:34:06.922917 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa4ca5-a9c3-4779-a825-502a22849540 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:06.923478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa4ca5-a9c3-4779-a825-502a22849540 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:06.923478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa4ca5-a9c3-4779-a825-502a22849540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:06.965553 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-bbaa4ca5-a9c3-4779-a825-502a22849540 None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:34:06.968845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bbaa4ca5-a9c3-4779-a825-502a22849540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:07.107337 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1121/2266] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 05:34:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:34:07.120802 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb85a4-012d-4a62-bfc6-943af3e3fdd8 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:07.122828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-13cb85a4-012d-4a62-bfc6-943af3e3fdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:07.233686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb85a4-012d-4a62-bfc6-943af3e3fdd8 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:07.244899 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce145d3-2e45-48b6-b3e7-2e2a60a5a4e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0330c79c391d463b91f99e030c6dcc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:07.245652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce145d3-2e45-48b6-b3e7-2e2a60a5a4e0 service glance] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:07.245968 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce145d3-2e45-48b6-b3e7-2e2a60a5a4e0 service glance] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:07.246294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce145d3-2e45-48b6-b3e7-2e2a60a5a4e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:07.266373 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce145d3-2e45-48b6-b3e7-2e2a60a5a4e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:07.267920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce145d3-2e45-48b6-b3e7-2e2a60a5a4e0 service glance] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:07.279946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1146/2267] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:34:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:34:07.393464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d464f-b1e5-4962-bc0c-759acb2c48c0 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:07.393923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d464f-b1e5-4962-bc0c-759acb2c48c0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:07.394302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d464f-b1e5-4962-bc0c-759acb2c48c0 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:07.396283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1122/2268] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 05:34:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:34:07.402880 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d8aac-854c-4502-9401-b12506a33aeb None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:07.403172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d8aac-854c-4502-9401-b12506a33aeb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:07.404237 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d8aac-854c-4502-9401-b12506a33aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:07.429851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ea4d8aac-854c-4502-9401-b12506a33aeb None None] MFA Rules not processed for user `0bf2a5de0e744505a1317ce9fa224643`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:34:07.431140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ea4d8aac-854c-4502-9401-b12506a33aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:07.469025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:34:07.503964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1147/2269] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 05:34:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:34:07.522364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d464f-b1e5-4962-bc0c-759acb2c48c0 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:07.524272 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f99d464f-b1e5-4962-bc0c-759acb2c48c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:07.533295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:34:07.550845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d464f-b1e5-4962-bc0c-759acb2c48c0 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:07.554925 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595a0bc-3f18-40e0-a397-70975ead6014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:07.555603 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595a0bc-3f18-40e0-a397-70975ead6014 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:07.555888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595a0bc-3f18-40e0-a397-70975ead6014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:07.556233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595a0bc-3f18-40e0-a397-70975ead6014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:07.570781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595a0bc-3f18-40e0-a397-70975ead6014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:07.570781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595a0bc-3f18-40e0-a397-70975ead6014 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:07.615300 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1123/2270] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:34:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:34:07.724727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ee7f-928d-4ca2-9135-04e1dd6a6917 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:07.725395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ee7f-928d-4ca2-9135-04e1dd6a6917 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:07.725832 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ee7f-928d-4ca2-9135-04e1dd6a6917 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:07.727389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1148/2271] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 05:34:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 05:34:07.739496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cae6bc-153f-47aa-8c6b-003e18faa307 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:07.739803 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cae6bc-153f-47aa-8c6b-003e18faa307 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:07.740090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cae6bc-153f-47aa-8c6b-003e18faa307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:07.763765 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-04cae6bc-153f-47aa-8c6b-003e18faa307 None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:34:07.765614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-04cae6bc-153f-47aa-8c6b-003e18faa307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:07.789012 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1124/2272] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 05:34:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:34:07.797869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7ee7f-928d-4ca2-9135-04e1dd6a6917 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:07.816499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7ee7f-928d-4ca2-9135-04e1dd6a6917 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:07.820495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fddcba-1d5c-4e2b-8559-563cb5439701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:07.821223 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fddcba-1d5c-4e2b-8559-563cb5439701 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:07.821637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fddcba-1d5c-4e2b-8559-563cb5439701 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:07.821993 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fddcba-1d5c-4e2b-8559-563cb5439701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:07.834781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fddcba-1d5c-4e2b-8559-563cb5439701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:07.836557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fddcba-1d5c-4e2b-8559-563cb5439701 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:07.848577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1149/2273] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:34:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:34:10.610837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595c1b7-59d6-45a4-a452-ca92d29c81d9 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:10.611403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595c1b7-59d6-45a4-a452-ca92d29c81d9 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:10.611630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595c1b7-59d6-45a4-a452-ca92d29c81d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:10.643406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a595c1b7-59d6-45a4-a452-ca92d29c81d9 None None] MFA Rules not processed for user `3e0d17c9b0e7488fbd1319658455e391`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:34:10.645225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a595c1b7-59d6-45a4-a452-ca92d29c81d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:10.721752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1125/2274] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:34:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:34:10.734274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fddcba-1d5c-4e2b-8559-563cb5439701 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:10.735738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-05fddcba-1d5c-4e2b-8559-563cb5439701 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:10.748053 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fddcba-1d5c-4e2b-8559-563cb5439701 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:10.753917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b310d21-3bbd-404c-b7c8-0e3fa8ef223a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0330c79c391d463b91f99e030c6dcc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:10.754618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b310d21-3bbd-404c-b7c8-0e3fa8ef223a service glance] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:10.754843 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b310d21-3bbd-404c-b7c8-0e3fa8ef223a service glance] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:10.755135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b310d21-3bbd-404c-b7c8-0e3fa8ef223a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:10.822635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b310d21-3bbd-404c-b7c8-0e3fa8ef223a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:10.824204 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b310d21-3bbd-404c-b7c8-0e3fa8ef223a service glance] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:10.834273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1150/2275] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:34:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:34:10.981861 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae694b-0b16-400d-b9b5-465a46b7c07f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:10.982153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae694b-0b16-400d-b9b5-465a46b7c07f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:10.982431 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae694b-0b16-400d-b9b5-465a46b7c07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:11.006127 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ccae694b-0b16-400d-b9b5-465a46b7c07f None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:34:11.007467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ccae694b-0b16-400d-b9b5-465a46b7c07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:11.088680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1126/2276] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 05:34:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:34:11.106378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b310d21-3bbd-404c-b7c8-0e3fa8ef223a service glance] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:11.107981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7b310d21-3bbd-404c-b7c8-0e3fa8ef223a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:11.211579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b310d21-3bbd-404c-b7c8-0e3fa8ef223a service glance] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:11.216744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f80ca4-2c77-48a4-b0db-2ec5a0b64799 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:11.217560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f80ca4-2c77-48a4-b0db-2ec5a0b64799 service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:11.217843 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f80ca4-2c77-48a4-b0db-2ec5a0b64799 service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:11.218279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f80ca4-2c77-48a4-b0db-2ec5a0b64799 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:11.227322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f80ca4-2c77-48a4-b0db-2ec5a0b64799 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:11.229130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f80ca4-2c77-48a4-b0db-2ec5a0b64799 service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:11.238424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1151/2277] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:34:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 05:34:19.322191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d23b14-9a00-4ab5-bdc8-f6a03babc630 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:19.322802 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d23b14-9a00-4ab5-bdc8-f6a03babc630 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:19.322802 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d23b14-9a00-4ab5-bdc8-f6a03babc630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:19.346814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d0d23b14-9a00-4ab5-bdc8-f6a03babc630 None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:34:19.348190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d0d23b14-9a00-4ab5-bdc8-f6a03babc630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:19.358701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1127/2278] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 05:34:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:34:19.374282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f80ca4-2c77-48a4-b0db-2ec5a0b64799 service swift] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:19.387346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f80ca4-2c77-48a4-b0db-2ec5a0b64799 service swift] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:19.394278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d723d9-630d-4a5e-9529-8c475a171b24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:19.394892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d723d9-630d-4a5e-9529-8c475a171b24 service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:19.395171 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d723d9-630d-4a5e-9529-8c475a171b24 service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:19.395440 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d723d9-630d-4a5e-9529-8c475a171b24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:19.404298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d723d9-630d-4a5e-9529-8c475a171b24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:19.406274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d723d9-630d-4a5e-9529-8c475a171b24 service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:19.414937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1152/2279] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:34:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 05:34:20.761054 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e554c88-45ff-4354-8841-3d404314848f None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:20.762015 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e554c88-45ff-4354-8841-3d404314848f None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:20.762288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e554c88-45ff-4354-8841-3d404314848f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:20.785860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4e554c88-45ff-4354-8841-3d404314848f None None] MFA Rules not processed for user `c9acda7a0d8c4834aace017083393cd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:34:20.787200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4e554c88-45ff-4354-8841-3d404314848f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:20.796695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1128/2280] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:34:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:34:29.411227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=82 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:34:30.778076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=73 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:34:40.069144 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e26aeb-f733-4eaa-8aba-cd59d6c3af2d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:40.069641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e26aeb-f733-4eaa-8aba-cd59d6c3af2d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:40.069754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e26aeb-f733-4eaa-8aba-cd59d6c3af2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:40.091660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e3e26aeb-f733-4eaa-8aba-cd59d6c3af2d None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:34:40.093113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e3e26aeb-f733-4eaa-8aba-cd59d6c3af2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:40.112176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1153/2281] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 05:34:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:34:40.116299 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e554c88-45ff-4354-8841-3d404314848f None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:40.129253 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e554c88-45ff-4354-8841-3d404314848f None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:40.135090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f269b8-aad0-4d7c-ada3-96bfe5b45572 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:40.135944 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f269b8-aad0-4d7c-ada3-96bfe5b45572 service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:40.136413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f269b8-aad0-4d7c-ada3-96bfe5b45572 service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:40.136841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f269b8-aad0-4d7c-ada3-96bfe5b45572 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:40.146781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f269b8-aad0-4d7c-ada3-96bfe5b45572 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:40.148408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f269b8-aad0-4d7c-ada3-96bfe5b45572 service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:40.156887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1129/2282] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:34:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 05:34:50.083633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:34:50.153220 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=6 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:34:50.774352 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e26aeb-f733-4eaa-8aba-cd59d6c3af2d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:50.784519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e26aeb-f733-4eaa-8aba-cd59d6c3af2d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:50.792350 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee81a16-835b-441e-af0f-f13db762ca56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:50.792959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81a16-835b-441e-af0f-f13db762ca56 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:50.793236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81a16-835b-441e-af0f-f13db762ca56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:50.793507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee81a16-835b-441e-af0f-f13db762ca56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:50.795760 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bee81a16-835b-441e-af0f-f13db762ca56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:50.809671 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee81a16-835b-441e-af0f-f13db762ca56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:50.812163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee81a16-835b-441e-af0f-f13db762ca56 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:50.823194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1154/2283] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:34:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:34:53.475071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f269b8-aad0-4d7c-ada3-96bfe5b45572 service swift] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:34:53.485419 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f269b8-aad0-4d7c-ada3-96bfe5b45572 service swift] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:34:53.489573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb751cb3-647f-418a-ae23-3be7b3176f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:34:53.490177 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb751cb3-647f-418a-ae23-3be7b3176f3b service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:34:53.490466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb751cb3-647f-418a-ae23-3be7b3176f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:34:53.490730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb751cb3-647f-418a-ae23-3be7b3176f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:34:53.492522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-fb751cb3-647f-418a-ae23-3be7b3176f3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:34:53.508221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb751cb3-647f-418a-ae23-3be7b3176f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:34:53.509934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb751cb3-647f-418a-ae23-3be7b3176f3b service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:34:53.522063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1130/2284] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:34:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:35:00.818340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=6 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:35:01.819203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee81a16-835b-441e-af0f-f13db762ca56 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:35:01.828579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee81a16-835b-441e-af0f-f13db762ca56 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:35:01.832202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64e9e6-239b-4d56-86e9-84446916314d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:35:01.832792 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64e9e6-239b-4d56-86e9-84446916314d service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:35:01.833067 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64e9e6-239b-4d56-86e9-84446916314d service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:35:01.833335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64e9e6-239b-4d56-86e9-84446916314d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:35:01.835197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-af64e9e6-239b-4d56-86e9-84446916314d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:35:01.909842 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64e9e6-239b-4d56-86e9-84446916314d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:35:01.911344 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64e9e6-239b-4d56-86e9-84446916314d service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:35:01.921787 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1155/2285] 10.222.0.33 () {60 vars in 1396 bytes} [Sun Sep 11 05:35:01 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:35:03.516785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=6 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:35:08.454008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242392a-6a1b-463b-ae43-9e2fa951a251 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:35:08.454697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242392a-6a1b-463b-ae43-9e2fa951a251 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:35:08.454697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242392a-6a1b-463b-ae43-9e2fa951a251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:35:08.476109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8242392a-6a1b-463b-ae43-9e2fa951a251 None None] MFA Rules not processed for user `9793a6256764417eb3ff1a8386368801`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:35:08.477345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8242392a-6a1b-463b-ae43-9e2fa951a251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:35:08.534663 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1131/2286] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:35:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:35:08.765906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64e9e6-239b-4d56-86e9-84446916314d service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:35:08.774886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64e9e6-239b-4d56-86e9-84446916314d service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:35:08.778891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb86e2-ae3a-4712-a697-5113ca924bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:35:08.779485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb86e2-ae3a-4712-a697-5113ca924bdf service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:35:08.779766 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb86e2-ae3a-4712-a697-5113ca924bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:35:08.780072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb86e2-ae3a-4712-a697-5113ca924bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:35:08.781805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0ceb86e2-ae3a-4712-a697-5113ca924bdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:35:08.791823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb86e2-ae3a-4712-a697-5113ca924bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:35:08.793281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb86e2-ae3a-4712-a697-5113ca924bdf service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:35:08.801726 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1156/2287] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:35:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:35:08.946791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242392a-6a1b-463b-ae43-9e2fa951a251 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:35:08.957870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242392a-6a1b-463b-ae43-9e2fa951a251 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:35:08.961815 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf321585-e048-4d0b-af8d-9085ea34b130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:35:08.962426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf321585-e048-4d0b-af8d-9085ea34b130 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:35:08.962704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf321585-e048-4d0b-af8d-9085ea34b130 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:35:08.962972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf321585-e048-4d0b-af8d-9085ea34b130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:35:08.964748 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bf321585-e048-4d0b-af8d-9085ea34b130 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:35:08.985429 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf321585-e048-4d0b-af8d-9085ea34b130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:35:08.987299 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf321585-e048-4d0b-af8d-9085ea34b130 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:35:08.997589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1132/2288] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:35:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:35:09.145641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb86e2-ae3a-4712-a697-5113ca924bdf service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:35:09.147263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0ceb86e2-ae3a-4712-a697-5113ca924bdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:35:09.241306 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb86e2-ae3a-4712-a697-5113ca924bdf service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:35:09.248538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffc337-1017-4d33-bfcd-cde38e7fe006 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:35:09.249034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffc337-1017-4d33-bfcd-cde38e7fe006 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:35:09.249347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffc337-1017-4d33-bfcd-cde38e7fe006 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:35:09.249588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffc337-1017-4d33-bfcd-cde38e7fe006 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:35:09.251433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-acffc337-1017-4d33-bfcd-cde38e7fe006 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:35:09.268659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffc337-1017-4d33-bfcd-cde38e7fe006 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:35:09.271485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffc337-1017-4d33-bfcd-cde38e7fe006 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:35:09.286824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1157/2289] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:35:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:35:12.569853 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdf9fe-a44d-4d7a-8c8f-e552c7cd7223 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:35:12.570292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdf9fe-a44d-4d7a-8c8f-e552c7cd7223 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:35:12.570423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdf9fe-a44d-4d7a-8c8f-e552c7cd7223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:35:12.604448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-13bdf9fe-a44d-4d7a-8c8f-e552c7cd7223 None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:35:12.606311 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-13bdf9fe-a44d-4d7a-8c8f-e552c7cd7223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:35:12.614524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1133/2290] 10.222.0.33 () {58 vars in 967 bytes} [Sun Sep 11 05:35:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:35:12.637978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffc337-1017-4d33-bfcd-cde38e7fe006 service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:35:12.649368 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffc337-1017-4d33-bfcd-cde38e7fe006 service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:35:12.655491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b9f06-7538-4cea-adbf-db32751432f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:35:12.656141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b9f06-7538-4cea-adbf-db32751432f3 service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:35:12.656454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b9f06-7538-4cea-adbf-db32751432f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:35:12.656821 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b9f06-7538-4cea-adbf-db32751432f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:35:12.665576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b9f06-7538-4cea-adbf-db32751432f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:35:12.667159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b9f06-7538-4cea-adbf-db32751432f3 service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:35:12.676398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1158/2291] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:35:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 05:35:17.095347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdf9fe-a44d-4d7a-8c8f-e552c7cd7223 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:35:17.097152 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-13bdf9fe-a44d-4d7a-8c8f-e552c7cd7223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:35:17.113893 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdf9fe-a44d-4d7a-8c8f-e552c7cd7223 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:35:17.119451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7a204-7270-4e90-808d-154204302fd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:35:17.120083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7a204-7270-4e90-808d-154204302fd0 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:35:17.120362 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7a204-7270-4e90-808d-154204302fd0 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:35:17.120648 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7a204-7270-4e90-808d-154204302fd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:35:17.122901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-26a7a204-7270-4e90-808d-154204302fd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:35:17.135462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7a204-7270-4e90-808d-154204302fd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:35:17.137071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7a204-7270-4e90-808d-154204302fd0 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:35:17.152070 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1134/2292] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:35:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:35:19.205869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b9f06-7538-4cea-adbf-db32751432f3 service swift] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:35:19.207973 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-eb7b9f06-7538-4cea-adbf-db32751432f3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:35:19.223646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b9f06-7538-4cea-adbf-db32751432f3 service swift] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:35:19.229117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770e026-c026-4f69-b10c-2b7793552f86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:35:19.229745 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770e026-c026-4f69-b10c-2b7793552f86 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:35:19.230025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770e026-c026-4f69-b10c-2b7793552f86 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:35:19.230301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770e026-c026-4f69-b10c-2b7793552f86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:35:19.232649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1770e026-c026-4f69-b10c-2b7793552f86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:35:19.291693 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770e026-c026-4f69-b10c-2b7793552f86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:35:19.293883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770e026-c026-4f69-b10c-2b7793552f86 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:35:19.298541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:35:19.303654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1159/2293] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:35:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:35:27.144072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=40 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:35:38.612775 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7a204-7270-4e90-808d-154204302fd0 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:35:38.622375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7a204-7270-4e90-808d-154204302fd0 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:35:38.626165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c2154-3bed-45c4-b599-8c5339cc30e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:35:38.626780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c2154-3bed-45c4-b599-8c5339cc30e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:35:38.627062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c2154-3bed-45c4-b599-8c5339cc30e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:35:38.627343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c2154-3bed-45c4-b599-8c5339cc30e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:35:38.637064 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c2154-3bed-45c4-b599-8c5339cc30e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:35:38.638521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c2154-3bed-45c4-b599-8c5339cc30e2 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:35:38.648304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1135/2294] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:35:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:35:40.419300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770e026-c026-4f69-b10c-2b7793552f86 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:35:40.434305 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770e026-c026-4f69-b10c-2b7793552f86 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:35:40.442137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2a202-6d48-4d17-a2dc-b1b6bfac94cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:35:40.443563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2a202-6d48-4d17-a2dc-b1b6bfac94cc service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:35:40.443699 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2a202-6d48-4d17-a2dc-b1b6bfac94cc service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:35:40.444515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2a202-6d48-4d17-a2dc-b1b6bfac94cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:35:40.446560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-12b2a202-6d48-4d17-a2dc-b1b6bfac94cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:35:40.461598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2a202-6d48-4d17-a2dc-b1b6bfac94cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:35:40.463340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b2a202-6d48-4d17-a2dc-b1b6bfac94cc service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:35:40.476173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1160/2295] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:35:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:35:48.643916 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=6 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:35:50.470649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=6 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:35:51.648478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c2154-3bed-45c4-b599-8c5339cc30e2 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:35:51.660929 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c2154-3bed-45c4-b599-8c5339cc30e2 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:35:51.666369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09501694-7878-420b-b870-7b345e2672f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:35:51.666954 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09501694-7878-420b-b870-7b345e2672f9 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:35:51.667229 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09501694-7878-420b-b870-7b345e2672f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:35:51.667497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09501694-7878-420b-b870-7b345e2672f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:35:51.669185 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-09501694-7878-420b-b870-7b345e2672f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:35:51.727098 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09501694-7878-420b-b870-7b345e2672f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:35:51.728705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09501694-7878-420b-b870-7b345e2672f9 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:35:51.737579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1136/2296] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:35:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:36:01.733434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=28 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:36:20.957943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2a202-6d48-4d17-a2dc-b1b6bfac94cc service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:36:20.969863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2a202-6d48-4d17-a2dc-b1b6bfac94cc service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:36:20.975449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0318dc3-c1bb-4d1d-b76d-5abca2f7992b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:36:20.976081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0318dc3-c1bb-4d1d-b76d-5abca2f7992b service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:36:20.976365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0318dc3-c1bb-4d1d-b76d-5abca2f7992b service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:36:20.976642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0318dc3-c1bb-4d1d-b76d-5abca2f7992b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:36:20.978365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f0318dc3-c1bb-4d1d-b76d-5abca2f7992b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:36:21.035453 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0318dc3-c1bb-4d1d-b76d-5abca2f7992b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:36:21.037076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0318dc3-c1bb-4d1d-b76d-5abca2f7992b service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:36:21.046493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1161/2297] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:36:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:36:31.041773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=28 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:36:50.397440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82652671-ca32-4cb1-81b7-3058d20175fc None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:36:50.397964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82652671-ca32-4cb1-81b7-3058d20175fc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:36:50.398062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82652671-ca32-4cb1-81b7-3058d20175fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:36:50.417269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-82652671-ca32-4cb1-81b7-3058d20175fc None None] MFA Rules not processed for user `f4c978e96d3248d59120c287ecc75e87`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:36:50.418465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-82652671-ca32-4cb1-81b7-3058d20175fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:36:50.425966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1137/2298] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:36:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:37:00.411662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:37:20.469191 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0318dc3-c1bb-4d1d-b76d-5abca2f7992b service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:37:20.480282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0318dc3-c1bb-4d1d-b76d-5abca2f7992b service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:37:20.486510 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e471852-bc45-4df6-9d8e-31019e74e9a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:37:20.487128 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e471852-bc45-4df6-9d8e-31019e74e9a4 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:37:20.487409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e471852-bc45-4df6-9d8e-31019e74e9a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:37:20.487681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e471852-bc45-4df6-9d8e-31019e74e9a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:37:20.489455 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1e471852-bc45-4df6-9d8e-31019e74e9a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:37:20.555460 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e471852-bc45-4df6-9d8e-31019e74e9a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:37:20.557088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e471852-bc45-4df6-9d8e-31019e74e9a4 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:37:20.567231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1162/2299] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:37:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:37:30.562203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=28 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:38:44.443750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82652671-ca32-4cb1-81b7-3058d20175fc None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:38:44.445168 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-82652671-ca32-4cb1-81b7-3058d20175fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:38:44.562316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82652671-ca32-4cb1-81b7-3058d20175fc None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:38:44.566027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216c3d6-83d7-4871-9cd2-f31f889dd33c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:38:44.566629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216c3d6-83d7-4871-9cd2-f31f889dd33c service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:38:44.566911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216c3d6-83d7-4871-9cd2-f31f889dd33c service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:38:44.567180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216c3d6-83d7-4871-9cd2-f31f889dd33c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:38:44.568949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c216c3d6-83d7-4871-9cd2-f31f889dd33c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:38:44.599838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216c3d6-83d7-4871-9cd2-f31f889dd33c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:38:44.601324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216c3d6-83d7-4871-9cd2-f31f889dd33c service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:38:44.611332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1138/2300] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:38:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:38:53.821723 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e471852-bc45-4df6-9d8e-31019e74e9a4 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:38:53.872286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e471852-bc45-4df6-9d8e-31019e74e9a4 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:38:53.877376 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00c5e0-00ec-496b-8281-080f9f6f7504 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:38:53.877971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c5e0-00ec-496b-8281-080f9f6f7504 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:38:53.878253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c5e0-00ec-496b-8281-080f9f6f7504 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:38:53.878529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c5e0-00ec-496b-8281-080f9f6f7504 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:38:53.880310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8d00c5e0-00ec-496b-8281-080f9f6f7504 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:38:53.904398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00c5e0-00ec-496b-8281-080f9f6f7504 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:38:53.905973 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00c5e0-00ec-496b-8281-080f9f6f7504 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:38:53.916422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1163/2301] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:38:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:38:54.606360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=54 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:39:03.912122 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=29 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:39:11.579956 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216c3d6-83d7-4871-9cd2-f31f889dd33c service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:39:11.608992 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216c3d6-83d7-4871-9cd2-f31f889dd33c service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:39:11.614819 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4dec4-6f25-4153-829e-8a3a1d65a1dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:39:11.615433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4dec4-6f25-4153-829e-8a3a1d65a1dc service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:39:11.615711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4dec4-6f25-4153-829e-8a3a1d65a1dc service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:39:11.615982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4dec4-6f25-4153-829e-8a3a1d65a1dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:39:11.623525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4dec4-6f25-4153-829e-8a3a1d65a1dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:39:11.625126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4dec4-6f25-4153-829e-8a3a1d65a1dc service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:39:11.634490 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1139/2302] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:39:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:39:21.630585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=13 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:40:44.811956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00c5e0-00ec-496b-8281-080f9f6f7504 service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:40:44.821486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00c5e0-00ec-496b-8281-080f9f6f7504 service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:40:44.825048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729577d-d88e-4833-8e6b-d014282a6fee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:40:44.825642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729577d-d88e-4833-8e6b-d014282a6fee service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:40:44.825924 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729577d-d88e-4833-8e6b-d014282a6fee service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:40:44.826196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729577d-d88e-4833-8e6b-d014282a6fee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:40:44.836282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729577d-d88e-4833-8e6b-d014282a6fee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:40:44.837689 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729577d-d88e-4833-8e6b-d014282a6fee service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:40:44.848943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1164/2303] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:40:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:40:44.855764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4dec4-6f25-4153-829e-8a3a1d65a1dc service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:40:44.865654 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4dec4-6f25-4153-829e-8a3a1d65a1dc service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:40:44.869215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e59438-9353-4090-a80f-c63c202a5a27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:40:44.869801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59438-9353-4090-a80f-c63c202a5a27 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:40:44.870080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59438-9353-4090-a80f-c63c202a5a27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:40:44.870349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59438-9353-4090-a80f-c63c202a5a27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:40:44.878704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e59438-9353-4090-a80f-c63c202a5a27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:40:44.880155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e59438-9353-4090-a80f-c63c202a5a27 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:40:44.906781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1140/2304] 10.222.0.33 () {60 vars in 1396 bytes} [Sun Sep 11 05:40:44 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3268 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:40:45.024879 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729577d-d88e-4833-8e6b-d014282a6fee service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:40:45.033762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729577d-d88e-4833-8e6b-d014282a6fee service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:40:45.038469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9574d-091e-4992-8c55-214ca2c15d81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:40:45.039157 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9574d-091e-4992-8c55-214ca2c15d81 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:40:45.039495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9574d-091e-4992-8c55-214ca2c15d81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:40:45.039833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9574d-091e-4992-8c55-214ca2c15d81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:40:45.051456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9574d-091e-4992-8c55-214ca2c15d81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:40:45.053069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9574d-091e-4992-8c55-214ca2c15d81 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:40:45.065053 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1165/2305] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:40:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:40:45.165231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e59438-9353-4090-a80f-c63c202a5a27 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:40:45.179905 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e59438-9353-4090-a80f-c63c202a5a27 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:40:45.191141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160473a0-1ca5-4fc6-af23-aedc4b3a3c1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:40:45.191771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160473a0-1ca5-4fc6-af23-aedc4b3a3c1c service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:40:45.192085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160473a0-1ca5-4fc6-af23-aedc4b3a3c1c service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:40:45.192378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160473a0-1ca5-4fc6-af23-aedc4b3a3c1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:40:45.200963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160473a0-1ca5-4fc6-af23-aedc4b3a3c1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:40:45.202545 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160473a0-1ca5-4fc6-af23-aedc4b3a3c1c service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:40:45.230321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1141/2306] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:40:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:40:52.504093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9574d-091e-4992-8c55-214ca2c15d81 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:40:52.522959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9574d-091e-4992-8c55-214ca2c15d81 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:40:52.526560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93176ee3-ae1c-400e-9b11-1141d897bd0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:40:52.527153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93176ee3-ae1c-400e-9b11-1141d897bd0e service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:40:52.527430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93176ee3-ae1c-400e-9b11-1141d897bd0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:40:52.527697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93176ee3-ae1c-400e-9b11-1141d897bd0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:40:52.534685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93176ee3-ae1c-400e-9b11-1141d897bd0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:40:52.536647 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93176ee3-ae1c-400e-9b11-1141d897bd0e service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:40:52.544376 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1166/2307] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:40:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:40:55.227440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=26 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:41:02.540942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=22 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:41:03.887676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160473a0-1ca5-4fc6-af23-aedc4b3a3c1c service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:41:03.889841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-160473a0-1ca5-4fc6-af23-aedc4b3a3c1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:41:03.902246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160473a0-1ca5-4fc6-af23-aedc4b3a3c1c service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:41:03.906354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e915c-328f-4f2d-b534-5e01e3e46dee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:41:03.907202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e915c-328f-4f2d-b534-5e01e3e46dee service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:41:03.907712 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e915c-328f-4f2d-b534-5e01e3e46dee service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:41:03.907987 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e915c-328f-4f2d-b534-5e01e3e46dee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:41:03.917762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e915c-328f-4f2d-b534-5e01e3e46dee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:41:03.919204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e915c-328f-4f2d-b534-5e01e3e46dee service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:41:03.949740 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1142/2308] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:41:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:41:13.946269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=13 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:41:20.497113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93176ee3-ae1c-400e-9b11-1141d897bd0e service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:41:20.510776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93176ee3-ae1c-400e-9b11-1141d897bd0e service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:41:20.516555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b1003-ff72-400f-a50c-7cf81192df92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:41:20.517151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b1003-ff72-400f-a50c-7cf81192df92 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:41:20.517434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b1003-ff72-400f-a50c-7cf81192df92 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:41:20.517707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b1003-ff72-400f-a50c-7cf81192df92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:41:20.525357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b1003-ff72-400f-a50c-7cf81192df92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:41:20.526936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b1003-ff72-400f-a50c-7cf81192df92 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:41:20.554134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1167/2309] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:41:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:41:30.550910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=13 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:42:45.306350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e915c-328f-4f2d-b534-5e01e3e46dee service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:42:45.316615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e915c-328f-4f2d-b534-5e01e3e46dee service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:42:45.322865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33349ca3-f766-4d4d-b358-97ef816cda38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:42:45.323471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33349ca3-f766-4d4d-b358-97ef816cda38 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:42:45.323752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33349ca3-f766-4d4d-b358-97ef816cda38 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:42:45.324048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33349ca3-f766-4d4d-b358-97ef816cda38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:42:45.331916 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33349ca3-f766-4d4d-b358-97ef816cda38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:42:45.333505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33349ca3-f766-4d4d-b358-97ef816cda38 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:42:45.362652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1143/2310] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:42:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:42:55.360051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=13 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:43:55.771437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b1003-ff72-400f-a50c-7cf81192df92 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:43:55.772987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2f5b1003-ff72-400f-a50c-7cf81192df92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:43:55.787057 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b1003-ff72-400f-a50c-7cf81192df92 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:43:55.792135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad008e8c-ef94-4deb-9288-d72a83747ade service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:43:55.792738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad008e8c-ef94-4deb-9288-d72a83747ade service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:43:55.793018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad008e8c-ef94-4deb-9288-d72a83747ade service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:43:55.793290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad008e8c-ef94-4deb-9288-d72a83747ade service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:43:55.802069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad008e8c-ef94-4deb-9288-d72a83747ade service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:43:55.803661 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad008e8c-ef94-4deb-9288-d72a83747ade service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:43:55.813575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1168/2311] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:43:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:44:05.809366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=6 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:44:45.506825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33349ca3-f766-4d4d-b358-97ef816cda38 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:44:45.516780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33349ca3-f766-4d4d-b358-97ef816cda38 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:44:45.520314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f4f17-1653-490a-ab7f-b6ecd0a55ec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:44:45.520904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f4f17-1653-490a-ab7f-b6ecd0a55ec1 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:44:45.521186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f4f17-1653-490a-ab7f-b6ecd0a55ec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:44:45.521465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f4f17-1653-490a-ab7f-b6ecd0a55ec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:44:45.530448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f4f17-1653-490a-ab7f-b6ecd0a55ec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:44:45.531909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f4f17-1653-490a-ab7f-b6ecd0a55ec1 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:44:45.559805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1144/2312] 10.222.0.33 () {60 vars in 1396 bytes} [Sun Sep 11 05:44:45 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:44:45.782314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad008e8c-ef94-4deb-9288-d72a83747ade service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:44:45.851384 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad008e8c-ef94-4deb-9288-d72a83747ade service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:44:45.857002 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a85c9e-ddf4-43a1-be5f-9573ed0ce007 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:44:45.857596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85c9e-ddf4-43a1-be5f-9573ed0ce007 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:44:45.857869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85c9e-ddf4-43a1-be5f-9573ed0ce007 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:44:45.858135 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a85c9e-ddf4-43a1-be5f-9573ed0ce007 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:44:45.859920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b0a85c9e-ddf4-43a1-be5f-9573ed0ce007 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:44:45.939044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a85c9e-ddf4-43a1-be5f-9573ed0ce007 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:44:45.940663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a85c9e-ddf4-43a1-be5f-9573ed0ce007 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:44:45.958350 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1169/2313] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:44:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:44:55.557292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=13 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:44:55.955768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=69 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:45:54.458386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f4f17-1653-490a-ab7f-b6ecd0a55ec1 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:45:54.460214 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f47f4f17-1653-490a-ab7f-b6ecd0a55ec1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:45:54.529223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f4f17-1653-490a-ab7f-b6ecd0a55ec1 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:45:54.532833 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a513c-1736-4b18-813d-97a47d5f80eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:45:54.533429 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a513c-1736-4b18-813d-97a47d5f80eb service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:45:54.533707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a513c-1736-4b18-813d-97a47d5f80eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:45:54.533983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a513c-1736-4b18-813d-97a47d5f80eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:45:54.535790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0c6a513c-1736-4b18-813d-97a47d5f80eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:45:54.545626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a513c-1736-4b18-813d-97a47d5f80eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:45:54.547051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a513c-1736-4b18-813d-97a47d5f80eb service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:45:54.554856 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1145/2314] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:45:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:46:04.551430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=31 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:46:45.712261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a85c9e-ddf4-43a1-be5f-9573ed0ce007 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:46:45.721979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a85c9e-ddf4-43a1-be5f-9573ed0ce007 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:46:45.725527 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33771009-6d0a-4060-84af-ec1eb4c2fe8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:46:45.726152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33771009-6d0a-4060-84af-ec1eb4c2fe8b service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:46:45.726430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33771009-6d0a-4060-84af-ec1eb4c2fe8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:46:45.726727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33771009-6d0a-4060-84af-ec1eb4c2fe8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:46:45.728557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-33771009-6d0a-4060-84af-ec1eb4c2fe8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:46:45.756059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33771009-6d0a-4060-84af-ec1eb4c2fe8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:46:45.757541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33771009-6d0a-4060-84af-ec1eb4c2fe8b service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:46:45.768867 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1170/2315] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:46:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:46:45.776138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a513c-1736-4b18-813d-97a47d5f80eb service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:46:45.785879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a513c-1736-4b18-813d-97a47d5f80eb service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:46:45.789496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507aa5a5-bf65-4a64-9c0f-a3ad7d2be9f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:46:45.790093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aa5a5-bf65-4a64-9c0f-a3ad7d2be9f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:46:45.790372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aa5a5-bf65-4a64-9c0f-a3ad7d2be9f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:46:45.790646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aa5a5-bf65-4a64-9c0f-a3ad7d2be9f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:46:45.792551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-507aa5a5-bf65-4a64-9c0f-a3ad7d2be9f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:46:45.869733 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507aa5a5-bf65-4a64-9c0f-a3ad7d2be9f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:46:45.871180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507aa5a5-bf65-4a64-9c0f-a3ad7d2be9f7 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:46:45.880785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1146/2316] 10.222.0.33 () {60 vars in 1396 bytes} [Sun Sep 11 05:46:45 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3268 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:46:45.924841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33771009-6d0a-4060-84af-ec1eb4c2fe8b service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:46:45.931873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33771009-6d0a-4060-84af-ec1eb4c2fe8b service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:46:45.936443 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427aa69-93e5-4c1a-b103-705c8e1cb8fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:46:45.937097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427aa69-93e5-4c1a-b103-705c8e1cb8fc service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:46:45.937383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427aa69-93e5-4c1a-b103-705c8e1cb8fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:46:45.937746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427aa69-93e5-4c1a-b103-705c8e1cb8fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:46:45.939647 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f427aa69-93e5-4c1a-b103-705c8e1cb8fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:46:45.954114 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427aa69-93e5-4c1a-b103-705c8e1cb8fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:46:45.955466 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427aa69-93e5-4c1a-b103-705c8e1cb8fc service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:46:45.966946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1171/2317] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:46:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:46:46.020895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507aa5a5-bf65-4a64-9c0f-a3ad7d2be9f7 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:46:46.021552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427aa69-93e5-4c1a-b103-705c8e1cb8fc service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:46:46.023097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f427aa69-93e5-4c1a-b103-705c8e1cb8fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:46:46.029561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507aa5a5-bf65-4a64-9c0f-a3ad7d2be9f7 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:46:46.034858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861521de-b428-42ee-85dd-512368585496 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:46:46.035752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861521de-b428-42ee-85dd-512368585496 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:46:46.036264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861521de-b428-42ee-85dd-512368585496 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:46:46.036669 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427aa69-93e5-4c1a-b103-705c8e1cb8fc service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:46:46.036962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861521de-b428-42ee-85dd-512368585496 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:46:46.039239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-861521de-b428-42ee-85dd-512368585496 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:46:46.042333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454544c-e2ea-4560-8b67-537add4976a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:46:46.042963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454544c-e2ea-4560-8b67-537add4976a2 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:46:46.043244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454544c-e2ea-4560-8b67-537add4976a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:46:46.043516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454544c-e2ea-4560-8b67-537add4976a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:46:46.045569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2454544c-e2ea-4560-8b67-537add4976a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:46:46.117827 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861521de-b428-42ee-85dd-512368585496 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:46:46.119260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861521de-b428-42ee-85dd-512368585496 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:46:46.119404 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454544c-e2ea-4560-8b67-537add4976a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:46:46.120871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454544c-e2ea-4560-8b67-537add4976a2 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:46:46.129269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1147/2318] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:46:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:46:46.130791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1172/2319] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:46:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:46:46.230479 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861521de-b428-42ee-85dd-512368585496 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:46:46.243835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861521de-b428-42ee-85dd-512368585496 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:46:46.250660 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a277c16-23b3-420d-ae09-2d8b9b681db1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:46:46.253770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a277c16-23b3-420d-ae09-2d8b9b681db1 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:46:46.253770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a277c16-23b3-420d-ae09-2d8b9b681db1 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:46:46.253770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a277c16-23b3-420d-ae09-2d8b9b681db1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:46:46.257794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5a277c16-23b3-420d-ae09-2d8b9b681db1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:46:46.329818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a277c16-23b3-420d-ae09-2d8b9b681db1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:46:46.331337 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a277c16-23b3-420d-ae09-2d8b9b681db1 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:46:46.341149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1148/2320] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:46:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:46:56.126596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=50 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:46:56.336384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:47:45.895347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454544c-e2ea-4560-8b67-537add4976a2 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:47:45.904462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454544c-e2ea-4560-8b67-537add4976a2 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:47:45.909154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac20d4-0160-4819-b7e5-87e5218758f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:47:45.909744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac20d4-0160-4819-b7e5-87e5218758f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:47:45.910020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac20d4-0160-4819-b7e5-87e5218758f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:47:45.910288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac20d4-0160-4819-b7e5-87e5218758f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:47:45.912081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-77ac20d4-0160-4819-b7e5-87e5218758f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:47:45.981971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac20d4-0160-4819-b7e5-87e5218758f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:47:45.983463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac20d4-0160-4819-b7e5-87e5218758f5 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:47:45.992351 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1173/2321] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:47:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:47:55.988304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=33 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:48:59.798144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a277c16-23b3-420d-ae09-2d8b9b681db1 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:48:59.800271 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5a277c16-23b3-420d-ae09-2d8b9b681db1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:48:59.944915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a277c16-23b3-420d-ae09-2d8b9b681db1 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:48:59.950274 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51201c6-2909-40ab-9b8f-64734bb423f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:48:59.950912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51201c6-2909-40ab-9b8f-64734bb423f7 service placement] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:48:59.951197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51201c6-2909-40ab-9b8f-64734bb423f7 service placement] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:48:59.951471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51201c6-2909-40ab-9b8f-64734bb423f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:48:59.953483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a51201c6-2909-40ab-9b8f-64734bb423f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:48:59.966024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51201c6-2909-40ab-9b8f-64734bb423f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:48:59.967683 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51201c6-2909-40ab-9b8f-64734bb423f7 service placement] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:48:59.978946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1149/2322] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:48:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:49:09.974268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=63 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:49:45.874286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac20d4-0160-4819-b7e5-87e5218758f5 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:49:45.875960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-77ac20d4-0160-4819-b7e5-87e5218758f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:49:45.971915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac20d4-0160-4819-b7e5-87e5218758f5 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:49:45.975568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caec9d-48b7-467b-956e-2d8c0535eaf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:49:45.976206 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caec9d-48b7-467b-956e-2d8c0535eaf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:49:45.976490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caec9d-48b7-467b-956e-2d8c0535eaf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:49:45.976762 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caec9d-48b7-467b-956e-2d8c0535eaf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:49:45.978596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b5caec9d-48b7-467b-956e-2d8c0535eaf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:49:46.047452 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5caec9d-48b7-467b-956e-2d8c0535eaf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:49:46.048920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5caec9d-48b7-467b-956e-2d8c0535eaf6 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:49:46.057548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1174/2323] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:49:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:49:46.262264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51201c6-2909-40ab-9b8f-64734bb423f7 service placement] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:49:46.272468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51201c6-2909-40ab-9b8f-64734bb423f7 service placement] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:49:46.276039 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1aa0b-b7b6-48e3-8dba-a66555759238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:49:46.276631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1aa0b-b7b6-48e3-8dba-a66555759238 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:49:46.276910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1aa0b-b7b6-48e3-8dba-a66555759238 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:49:46.277180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1aa0b-b7b6-48e3-8dba-a66555759238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:49:46.285569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1aa0b-b7b6-48e3-8dba-a66555759238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:49:46.287042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1aa0b-b7b6-48e3-8dba-a66555759238 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:49:46.296130 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1150/2324] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:49:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:49:56.053554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=71 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:49:56.291941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=6 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:50:56.482055 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caec9d-48b7-467b-956e-2d8c0535eaf6 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:50:56.500768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caec9d-48b7-467b-956e-2d8c0535eaf6 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:50:56.504364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cca5c5-cc35-4b04-b2ee-6f1550facd09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:50:56.504959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cca5c5-cc35-4b04-b2ee-6f1550facd09 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:50:56.505235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cca5c5-cc35-4b04-b2ee-6f1550facd09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:50:56.505508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cca5c5-cc35-4b04-b2ee-6f1550facd09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:50:56.513011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cca5c5-cc35-4b04-b2ee-6f1550facd09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:50:56.514426 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cca5c5-cc35-4b04-b2ee-6f1550facd09 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:50:56.522857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1175/2325] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:50:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:51:06.519044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=10 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:51:45.926077 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1aa0b-b7b6-48e3-8dba-a66555759238 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:51:45.956027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1aa0b-b7b6-48e3-8dba-a66555759238 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:51:45.961449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d11691-d03c-467e-be82-0bed32e87027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:51:45.962076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d11691-d03c-467e-be82-0bed32e87027 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:51:45.962354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d11691-d03c-467e-be82-0bed32e87027 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:51:45.962624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d11691-d03c-467e-be82-0bed32e87027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:51:45.971631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d11691-d03c-467e-be82-0bed32e87027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:51:45.973231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d11691-d03c-467e-be82-0bed32e87027 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:51:46.000881 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1151/2326] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:51:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:51:46.043550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cca5c5-cc35-4b04-b2ee-6f1550facd09 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:51:46.045859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e5cca5c5-cc35-4b04-b2ee-6f1550facd09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:51:46.062614 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cca5c5-cc35-4b04-b2ee-6f1550facd09 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:51:46.067937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f4470-785a-42f2-9dd8-a1cac2fa743c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:51:46.068561 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f4470-785a-42f2-9dd8-a1cac2fa743c service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:51:46.068837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f4470-785a-42f2-9dd8-a1cac2fa743c service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:51:46.069110 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f4470-785a-42f2-9dd8-a1cac2fa743c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:51:46.077151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f4470-785a-42f2-9dd8-a1cac2fa743c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:51:46.078704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f4470-785a-42f2-9dd8-a1cac2fa743c service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:51:46.108100 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1176/2327] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:51:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:51:46.159042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d11691-d03c-467e-be82-0bed32e87027 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:51:46.168852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d11691-d03c-467e-be82-0bed32e87027 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:51:46.172363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddf426-db17-463d-906a-654c3c3a6960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:51:46.172943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddf426-db17-463d-906a-654c3c3a6960 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:51:46.173218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddf426-db17-463d-906a-654c3c3a6960 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:51:46.173487 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddf426-db17-463d-906a-654c3c3a6960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:51:46.183418 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddf426-db17-463d-906a-654c3c3a6960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:51:46.184859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cddf426-db17-463d-906a-654c3c3a6960 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:51:46.195357 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1152/2328] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:51:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:51:46.654910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f4470-785a-42f2-9dd8-a1cac2fa743c service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:51:46.664124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f4470-785a-42f2-9dd8-a1cac2fa743c service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:51:46.667728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabf10f-cfd1-4ebd-8e00-37bd00226b47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:51:46.668355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabf10f-cfd1-4ebd-8e00-37bd00226b47 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:51:46.668635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabf10f-cfd1-4ebd-8e00-37bd00226b47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:51:46.668904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabf10f-cfd1-4ebd-8e00-37bd00226b47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:51:46.677146 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabf10f-cfd1-4ebd-8e00-37bd00226b47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:51:46.678557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabf10f-cfd1-4ebd-8e00-37bd00226b47 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:51:46.706160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1177/2329] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:51:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:51:46.794784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddf426-db17-463d-906a-654c3c3a6960 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:51:46.804087 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddf426-db17-463d-906a-654c3c3a6960 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:51:46.808444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652fff5-42c2-42b8-88b2-1683a9a16778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:51:46.809040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652fff5-42c2-42b8-88b2-1683a9a16778 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:51:46.809317 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652fff5-42c2-42b8-88b2-1683a9a16778 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:51:46.809588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652fff5-42c2-42b8-88b2-1683a9a16778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:51:46.818156 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652fff5-42c2-42b8-88b2-1683a9a16778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:51:46.819568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652fff5-42c2-42b8-88b2-1683a9a16778 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:51:46.847593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1153/2330] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:51:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:51:56.703592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=26 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:51:56.844974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=39 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:52:46.108559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabf10f-cfd1-4ebd-8e00-37bd00226b47 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:52:46.116886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabf10f-cfd1-4ebd-8e00-37bd00226b47 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:52:46.123076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190fed7-2398-40c1-a0d2-fc8060de5bdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:52:46.124051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190fed7-2398-40c1-a0d2-fc8060de5bdf service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:52:46.124450 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190fed7-2398-40c1-a0d2-fc8060de5bdf service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:52:46.124755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190fed7-2398-40c1-a0d2-fc8060de5bdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:52:46.133142 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190fed7-2398-40c1-a0d2-fc8060de5bdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:52:46.134558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190fed7-2398-40c1-a0d2-fc8060de5bdf service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:52:46.162137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1178/2331] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:52:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:52:56.159466 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=13 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:53:06.665312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652fff5-42c2-42b8-88b2-1683a9a16778 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:53:06.677007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652fff5-42c2-42b8-88b2-1683a9a16778 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:53:06.681112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26135c6-09f4-4477-a29c-008bdeac62ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:53:06.689220 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26135c6-09f4-4477-a29c-008bdeac62ff service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:53:06.689578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26135c6-09f4-4477-a29c-008bdeac62ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:53:06.691108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26135c6-09f4-4477-a29c-008bdeac62ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:53:06.694746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e26135c6-09f4-4477-a29c-008bdeac62ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:53:06.743710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190fed7-2398-40c1-a0d2-fc8060de5bdf service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:53:06.745607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2190fed7-2398-40c1-a0d2-fc8060de5bdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:53:06.759330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190fed7-2398-40c1-a0d2-fc8060de5bdf service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:53:06.763853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beff149-d8c5-4cde-8f88-50c4e971307e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:53:06.764701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beff149-d8c5-4cde-8f88-50c4e971307e service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:53:06.764987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beff149-d8c5-4cde-8f88-50c4e971307e service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:53:06.765258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beff149-d8c5-4cde-8f88-50c4e971307e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:53:06.776232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beff149-d8c5-4cde-8f88-50c4e971307e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:53:06.777654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beff149-d8c5-4cde-8f88-50c4e971307e service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:53:06.789833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1179/2332] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:53:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:53:06.798044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26135c6-09f4-4477-a29c-008bdeac62ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:53:06.799530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26135c6-09f4-4477-a29c-008bdeac62ff service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:53:06.825915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1154/2333] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:53:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:53:10.320962 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beff149-d8c5-4cde-8f88-50c4e971307e service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:53:10.340312 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beff149-d8c5-4cde-8f88-50c4e971307e service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:53:10.350479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236998a5-876f-43f8-9804-8c33b06bf775 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:53:10.352074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236998a5-876f-43f8-9804-8c33b06bf775 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:53:10.352526 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236998a5-876f-43f8-9804-8c33b06bf775 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:53:10.353098 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236998a5-876f-43f8-9804-8c33b06bf775 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:53:10.355464 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-236998a5-876f-43f8-9804-8c33b06bf775 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:53:10.375169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236998a5-876f-43f8-9804-8c33b06bf775 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:53:10.376805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236998a5-876f-43f8-9804-8c33b06bf775 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:53:10.390321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1180/2334] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:53:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:53:14.195842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26135c6-09f4-4477-a29c-008bdeac62ff service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:53:14.208747 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26135c6-09f4-4477-a29c-008bdeac62ff service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:53:14.214042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031e5e1-e1a8-4098-b689-10fe77aff772 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:53:14.215053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031e5e1-e1a8-4098-b689-10fe77aff772 service placement] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:53:14.215402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031e5e1-e1a8-4098-b689-10fe77aff772 service placement] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:53:14.217859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031e5e1-e1a8-4098-b689-10fe77aff772 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:53:14.220162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1031e5e1-e1a8-4098-b689-10fe77aff772 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:53:14.237374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031e5e1-e1a8-4098-b689-10fe77aff772 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:53:14.239947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031e5e1-e1a8-4098-b689-10fe77aff772 service placement] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:53:14.262891 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1155/2335] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:53:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:53:14.273248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236998a5-876f-43f8-9804-8c33b06bf775 service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:53:14.275603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-236998a5-876f-43f8-9804-8c33b06bf775 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:53:14.292933 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236998a5-876f-43f8-9804-8c33b06bf775 service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:53:14.299491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:53:14.300528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:53:14.300811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:53:14.301088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] PATH_INFO: `/v3/users/25cdba6a3bec49b4928b3fd953673060` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:53:14.302258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25cdba6a3bec49b4928b3fd953673060)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:53:14.304227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:53:14.357203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25cdba6a3bec49b4928b3fd953673060'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.357804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25cdba6a3bec49b4928b3fd953673060'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.364836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25cdba6a3bec49b4928b3fd953673060'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.375034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25cdba6a3bec49b4928b3fd953673060'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.391024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25cdba6a3bec49b4928b3fd953673060'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.397839 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1181/2336] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:53:14 2022] DELETE /identity/v3/users/25cdba6a3bec49b4928b3fd953673060 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:53:14.570116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031e5e1-e1a8-4098-b689-10fe77aff772 service placement] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:53:14.591782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031e5e1-e1a8-4098-b689-10fe77aff772 service placement] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:53:14.668110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:53:14.669026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:53:14.669534 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:53:14.669830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] PATH_INFO: `/v3/projects/183fd2ebaf054cfebad0bafa207f9967` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:53:14.671007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=183fd2ebaf054cfebad0bafa207f9967)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:53:14.673046 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:53:14.722172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '183fd2ebaf054cfebad0bafa207f9967', 'request_id': 'req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.727039 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:53:14.732238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '183fd2ebaf054cfebad0bafa207f9967', 'request_id': 'req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.739983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '183fd2ebaf054cfebad0bafa207f9967', 'request_id': 'req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.751217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] The token cache is being invalidate because project 183fd2ebaf054cfebad0bafa207f9967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:53:14.751455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.752134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.753629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1156/2337] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:53:14 2022] DELETE /identity/v3/projects/183fd2ebaf054cfebad0bafa207f9967 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:53:14.757692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:53:14.759091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:53:14.772833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02dc45-30ab-4fd6-b3d2-82cb86db3285 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:53:14.836334 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:53:14.836968 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:53:14.837247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:53:14.837520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] PATH_INFO: `/v3/users/a53200c02ebf45589ae1c2e080949363` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:53:14.851319 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a53200c02ebf45589ae1c2e080949363)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:53:14.853310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:53:14.891824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:53:14.904919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a53200c02ebf45589ae1c2e080949363'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.905586 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a53200c02ebf45589ae1c2e080949363'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.912960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a53200c02ebf45589ae1c2e080949363'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.921891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a53200c02ebf45589ae1c2e080949363'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.934363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a53200c02ebf45589ae1c2e080949363'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:14.940294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1182/2338] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 05:53:14 2022] DELETE /identity/v3/users/a53200c02ebf45589ae1c2e080949363 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:53:15.113807 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:53:15.117537 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:53:15.131517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2cf3f-3f61-4bd3-9730-7e338c8b0751 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:53:15.215531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:53:15.216312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:53:15.216601 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:53:15.216860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] PATH_INFO: `/v3/projects/d1072ad00e8d4793b264b6d4a8185c3b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:53:15.227814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1072ad00e8d4793b264b6d4a8185c3b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:53:15.229792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:53:15.276086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1072ad00e8d4793b264b6d4a8185c3b', 'request_id': 'req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:15.283546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1072ad00e8d4793b264b6d4a8185c3b', 'request_id': 'req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:15.290384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1072ad00e8d4793b264b6d4a8185c3b', 'request_id': 'req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:15.301946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] The token cache is being invalidate because project d1072ad00e8d4793b264b6d4a8185c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 05:53:15.302239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:15.303284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 05:53:15.304834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1157/2339] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 05:53:15 2022] DELETE /identity/v3/projects/d1072ad00e8d4793b264b6d4a8185c3b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 05:53:24.937049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=15,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:53:25.298661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=61,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:54:03.751834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:54:03.753369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:54:03.830650 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed78f59-4d7f-4ee1-8963-614749c77940 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:54:03.889892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb91ba-f666-4684-968c-b6024822ae68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:54:03.890503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb91ba-f666-4684-968c-b6024822ae68 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:54:03.890779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb91ba-f666-4684-968c-b6024822ae68 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:54:03.891303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb91ba-f666-4684-968c-b6024822ae68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:54:03.893107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dfcb91ba-f666-4684-968c-b6024822ae68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:54:03.904529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb91ba-f666-4684-968c-b6024822ae68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:54:03.906117 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb91ba-f666-4684-968c-b6024822ae68 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:54:03.916102 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1183/2340] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:54:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:54:13.912348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=58 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:54:44.981222 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:54:44.982635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:54:45.055508 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef30d2-ab61-44f9-9f16-83e2d84ae6e0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:54:45.061371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7b846-753b-4589-ab01-799dfb922380 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:54:45.061909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b846-753b-4589-ab01-799dfb922380 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:54:45.062186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b846-753b-4589-ab01-799dfb922380 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:54:45.062459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b846-753b-4589-ab01-799dfb922380 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:54:45.064172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-68c7b846-753b-4589-ab01-799dfb922380 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:54:45.095317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb91ba-f666-4684-968c-b6024822ae68 service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:54:45.096722 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dfcb91ba-f666-4684-968c-b6024822ae68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:54:45.111906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb91ba-f666-4684-968c-b6024822ae68 service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:54:45.117490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f4c0d-6dc9-462f-b99c-42798a13c717 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:54:45.118091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f4c0d-6dc9-462f-b99c-42798a13c717 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:54:45.118383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f4c0d-6dc9-462f-b99c-42798a13c717 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:54:45.118657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f4c0d-6dc9-462f-b99c-42798a13c717 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:54:45.120414 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-573f4c0d-6dc9-462f-b99c-42798a13c717 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:54:45.123252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7b846-753b-4589-ab01-799dfb922380 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:54:45.124901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7b846-753b-4589-ab01-799dfb922380 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:54:45.134631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1158/2341] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:54:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:54:45.194481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f4c0d-6dc9-462f-b99c-42798a13c717 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:54:45.196183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f4c0d-6dc9-462f-b99c-42798a13c717 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:54:45.212054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1184/2342] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:54:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:54:55.130436 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=56 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:54:55.202499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=28 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:55:06.742437 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7b846-753b-4589-ab01-799dfb922380 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:55:06.758902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7b846-753b-4589-ab01-799dfb922380 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:55:06.765769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186f404-4465-4e36-8051-97f3c0cc0775 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:55:06.766772 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186f404-4465-4e36-8051-97f3c0cc0775 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:55:06.767216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186f404-4465-4e36-8051-97f3c0cc0775 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:55:06.767566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186f404-4465-4e36-8051-97f3c0cc0775 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:55:06.769883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f186f404-4465-4e36-8051-97f3c0cc0775 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:55:06.789969 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186f404-4465-4e36-8051-97f3c0cc0775 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:55:06.792513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186f404-4465-4e36-8051-97f3c0cc0775 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:55:06.811686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1159/2343] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:55:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 05:55:16.804401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=6 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:56:01.452832 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f4c0d-6dc9-462f-b99c-42798a13c717 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:56:01.454633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-573f4c0d-6dc9-462f-b99c-42798a13c717 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:01.525094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f4c0d-6dc9-462f-b99c-42798a13c717 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:56:01.529001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fa54e-205e-433a-81ab-fb627fd0061b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:56:01.529601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fa54e-205e-433a-81ab-fb627fd0061b service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:56:01.529876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fa54e-205e-433a-81ab-fb627fd0061b service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:56:01.530146 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fa54e-205e-433a-81ab-fb627fd0061b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:56:01.531838 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bc7fa54e-205e-433a-81ab-fb627fd0061b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:01.540953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7fa54e-205e-433a-81ab-fb627fd0061b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:56:01.542376 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7fa54e-205e-433a-81ab-fb627fd0061b service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:56:01.551243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1185/2344] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:56:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:56:11.547595 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=31 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:56:46.299824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186f404-4465-4e36-8051-97f3c0cc0775 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:56:46.311596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186f404-4465-4e36-8051-97f3c0cc0775 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:56:46.317129 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b68dbc-b9b5-4034-9afa-da84c7ad5b3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:56:46.317721 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b68dbc-b9b5-4034-9afa-da84c7ad5b3d service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:56:46.318001 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b68dbc-b9b5-4034-9afa-da84c7ad5b3d service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:56:46.318272 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b68dbc-b9b5-4034-9afa-da84c7ad5b3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:56:46.319987 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-70b68dbc-b9b5-4034-9afa-da84c7ad5b3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:46.388824 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b68dbc-b9b5-4034-9afa-da84c7ad5b3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:56:46.390351 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b68dbc-b9b5-4034-9afa-da84c7ad5b3d service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:56:46.399702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1160/2345] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:56:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:56:46.422956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fa54e-205e-433a-81ab-fb627fd0061b service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:56:46.436427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fa54e-205e-433a-81ab-fb627fd0061b service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:56:46.441914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d2581-684c-467d-b2b0-d5ee3c824390 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:56:46.442519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d2581-684c-467d-b2b0-d5ee3c824390 service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:56:46.442801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d2581-684c-467d-b2b0-d5ee3c824390 service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:56:46.443075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d2581-684c-467d-b2b0-d5ee3c824390 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:56:46.444985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ac7d2581-684c-467d-b2b0-d5ee3c824390 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:46.451754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b68dbc-b9b5-4034-9afa-da84c7ad5b3d service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:56:46.461298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b68dbc-b9b5-4034-9afa-da84c7ad5b3d service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:56:46.465328 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243c98b-99bb-4674-8dea-7dd2053a3abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:56:46.465921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c98b-99bb-4674-8dea-7dd2053a3abc service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:56:46.466198 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c98b-99bb-4674-8dea-7dd2053a3abc service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:56:46.466470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c98b-99bb-4674-8dea-7dd2053a3abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:56:46.468215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a243c98b-99bb-4674-8dea-7dd2053a3abc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:46.496909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243c98b-99bb-4674-8dea-7dd2053a3abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:56:46.498350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243c98b-99bb-4674-8dea-7dd2053a3abc service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:56:46.509335 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1161/2346] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:56:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:56:46.516952 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d2581-684c-467d-b2b0-d5ee3c824390 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:56:46.518806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d2581-684c-467d-b2b0-d5ee3c824390 service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:56:46.529645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1186/2347] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:56:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:56:46.774485 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243c98b-99bb-4674-8dea-7dd2053a3abc service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:56:46.775872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a243c98b-99bb-4674-8dea-7dd2053a3abc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:46.868117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243c98b-99bb-4674-8dea-7dd2053a3abc service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:56:46.872949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3680101-2441-44a8-b71e-c73fae04e910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:56:46.873638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3680101-2441-44a8-b71e-c73fae04e910 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:56:46.874039 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3680101-2441-44a8-b71e-c73fae04e910 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:56:46.874386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3680101-2441-44a8-b71e-c73fae04e910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:56:46.891648 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f3680101-2441-44a8-b71e-c73fae04e910 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:46.969961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3680101-2441-44a8-b71e-c73fae04e910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:56:46.971370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3680101-2441-44a8-b71e-c73fae04e910 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:56:46.981271 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1162/2348] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:56:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:56:47.714501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d2581-684c-467d-b2b0-d5ee3c824390 service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:56:47.724485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d2581-684c-467d-b2b0-d5ee3c824390 service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:56:47.728213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b778ed9-d177-4a49-b7ac-c229dfe0c7ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:56:47.728945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b778ed9-d177-4a49-b7ac-c229dfe0c7ea service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:56:47.729294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b778ed9-d177-4a49-b7ac-c229dfe0c7ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:56:47.729574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b778ed9-d177-4a49-b7ac-c229dfe0c7ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:56:47.731593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8b778ed9-d177-4a49-b7ac-c229dfe0c7ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:47.812331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b778ed9-d177-4a49-b7ac-c229dfe0c7ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:56:47.813960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b778ed9-d177-4a49-b7ac-c229dfe0c7ea service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:56:47.824103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1187/2349] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:56:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:56:49.379442 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3680101-2441-44a8-b71e-c73fae04e910 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:56:49.397117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3680101-2441-44a8-b71e-c73fae04e910 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:56:49.403670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613a213-ec46-41c5-8949-742640b2e459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:56:49.404123 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613a213-ec46-41c5-8949-742640b2e459 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:56:49.404579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613a213-ec46-41c5-8949-742640b2e459 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:56:49.404865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613a213-ec46-41c5-8949-742640b2e459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:56:49.416112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9613a213-ec46-41c5-8949-742640b2e459 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:49.424713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:56:49.430582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613a213-ec46-41c5-8949-742640b2e459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:56:49.432643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613a213-ec46-41c5-8949-742640b2e459 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:56:49.449395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1163/2350] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:56:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:56:50.102734 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14360ad-4c1c-4954-a7b7-c7217e4f974e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:56:50.103247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14360ad-4c1c-4954-a7b7-c7217e4f974e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:56:50.103366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14360ad-4c1c-4954-a7b7-c7217e4f974e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:56:50.131073 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f14360ad-4c1c-4954-a7b7-c7217e4f974e None None] MFA Rules not processed for user `0d1dbe40cb994491bafc6ce1f947f4f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:56:50.132318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f14360ad-4c1c-4954-a7b7-c7217e4f974e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:50.139919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1188/2351] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:56:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:56:50.452091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613a213-ec46-41c5-8949-742640b2e459 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:56:50.462124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613a213-ec46-41c5-8949-742640b2e459 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:56:50.465865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf512f-c857-46cf-be39-aa0ad1013d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:56:50.466552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf512f-c857-46cf-be39-aa0ad1013d64 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:56:50.466899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf512f-c857-46cf-be39-aa0ad1013d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:56:50.467236 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf512f-c857-46cf-be39-aa0ad1013d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:56:50.469162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4fbf512f-c857-46cf-be39-aa0ad1013d64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:50.478841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf512f-c857-46cf-be39-aa0ad1013d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:56:50.480772 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf512f-c857-46cf-be39-aa0ad1013d64 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:56:50.489752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1164/2352] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:56:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:56:50.827416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14360ad-4c1c-4954-a7b7-c7217e4f974e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:56:50.828931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f14360ad-4c1c-4954-a7b7-c7217e4f974e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:50.857129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14360ad-4c1c-4954-a7b7-c7217e4f974e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:56:50.862534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666e8e3-8889-4769-b666-bab9726c5d66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:56:50.863143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666e8e3-8889-4769-b666-bab9726c5d66 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:56:50.863610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666e8e3-8889-4769-b666-bab9726c5d66 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:56:50.864033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666e8e3-8889-4769-b666-bab9726c5d66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:56:50.865765 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8666e8e3-8889-4769-b666-bab9726c5d66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:50.879377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666e8e3-8889-4769-b666-bab9726c5d66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:56:50.880998 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666e8e3-8889-4769-b666-bab9726c5d66 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:56:50.891436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1189/2353] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:56:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:56:59.319264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf512f-c857-46cf-be39-aa0ad1013d64 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:56:59.334736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf512f-c857-46cf-be39-aa0ad1013d64 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:56:59.341164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192ca26-b09d-4c17-a331-ed946e5d32f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:56:59.341877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192ca26-b09d-4c17-a331-ed946e5d32f3 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:56:59.342162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192ca26-b09d-4c17-a331-ed946e5d32f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:56:59.342437 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192ca26-b09d-4c17-a331-ed946e5d32f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:56:59.344192 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c192ca26-b09d-4c17-a331-ed946e5d32f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:56:59.356939 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192ca26-b09d-4c17-a331-ed946e5d32f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:56:59.358549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192ca26-b09d-4c17-a331-ed946e5d32f3 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:56:59.369548 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1165/2354] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:56:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:57:00.887068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=78 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:57:09.364844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=15 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:58:06.374186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666e8e3-8889-4769-b666-bab9726c5d66 service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:58:06.375742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8666e8e3-8889-4769-b666-bab9726c5d66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:58:06.387126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666e8e3-8889-4769-b666-bab9726c5d66 service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:58:06.391353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5e1e8-9ce0-472c-b0a1-c8bfcc72e9eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:58:06.391958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5e1e8-9ce0-472c-b0a1-c8bfcc72e9eb service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:58:06.392266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5e1e8-9ce0-472c-b0a1-c8bfcc72e9eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:58:06.392805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5e1e8-9ce0-472c-b0a1-c8bfcc72e9eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:58:06.394534 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-55a5e1e8-9ce0-472c-b0a1-c8bfcc72e9eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:58:06.408992 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5e1e8-9ce0-472c-b0a1-c8bfcc72e9eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:58:06.410421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5e1e8-9ce0-472c-b0a1-c8bfcc72e9eb service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:58:06.421654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1190/2355] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:58:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:58:16.416404 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=6 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:59:06.430506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781ccc8-ccdd-438b-b284-38fc6ca0e821 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:59:06.430930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781ccc8-ccdd-438b-b284-38fc6ca0e821 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:59:06.431086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781ccc8-ccdd-438b-b284-38fc6ca0e821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:59:06.491916 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a781ccc8-ccdd-438b-b284-38fc6ca0e821 None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:59:06.493136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a781ccc8-ccdd-438b-b284-38fc6ca0e821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:59:06.566064 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1166/2356] 10.222.0.33 () {58 vars in 967 bytes} [Sun Sep 11 05:59:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:59:06.613076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5e1e8-9ce0-472c-b0a1-c8bfcc72e9eb service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:59:06.614485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-55a5e1e8-9ce0-472c-b0a1-c8bfcc72e9eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:59:06.732152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5e1e8-9ce0-472c-b0a1-c8bfcc72e9eb service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:59:06.737212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e773421-9d4d-44b1-88dc-fc34f879263f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:59:06.737811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e773421-9d4d-44b1-88dc-fc34f879263f service swift] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:59:06.738094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e773421-9d4d-44b1-88dc-fc34f879263f service swift] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:59:06.738370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e773421-9d4d-44b1-88dc-fc34f879263f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:59:06.746262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e773421-9d4d-44b1-88dc-fc34f879263f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:59:06.747868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e773421-9d4d-44b1-88dc-fc34f879263f service swift] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:59:06.755717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1191/2357] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 05:59:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 05:59:07.369099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781ccc8-ccdd-438b-b284-38fc6ca0e821 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:59:07.386982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781ccc8-ccdd-438b-b284-38fc6ca0e821 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:59:07.397182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c03bd-1fa6-4719-b79a-7b6d5b3e5fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:59:07.397979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c03bd-1fa6-4719-b79a-7b6d5b3e5fac service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:59:07.398382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c03bd-1fa6-4719-b79a-7b6d5b3e5fac service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:59:07.398762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c03bd-1fa6-4719-b79a-7b6d5b3e5fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:59:07.402114 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-559c03bd-1fa6-4719-b79a-7b6d5b3e5fac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:59:07.493740 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c03bd-1fa6-4719-b79a-7b6d5b3e5fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:59:07.495243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c03bd-1fa6-4719-b79a-7b6d5b3e5fac service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:59:07.505506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1167/2358] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:59:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:59:09.823852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e773421-9d4d-44b1-88dc-fc34f879263f service swift] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:59:09.868768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e773421-9d4d-44b1-88dc-fc34f879263f service swift] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:59:09.876807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56c816-b83d-43d3-8de7-953de39d8ad7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:59:09.877494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56c816-b83d-43d3-8de7-953de39d8ad7 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:59:09.877846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56c816-b83d-43d3-8de7-953de39d8ad7 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:59:09.878212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56c816-b83d-43d3-8de7-953de39d8ad7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:59:09.893174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56c816-b83d-43d3-8de7-953de39d8ad7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:59:09.899377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56c816-b83d-43d3-8de7-953de39d8ad7 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:59:09.948235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1192/2359] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:59:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:59:12.273858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feacfe3-1735-4f2a-b710-7d0c86c3d021 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:59:12.274353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feacfe3-1735-4f2a-b710-7d0c86c3d021 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:59:12.274446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feacfe3-1735-4f2a-b710-7d0c86c3d021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:59:12.302799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0feacfe3-1735-4f2a-b710-7d0c86c3d021 None None] MFA Rules not processed for user `54b72a430d9d4d43954eb56d703079b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 05:59:12.304033 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0feacfe3-1735-4f2a-b710-7d0c86c3d021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:59:12.311120 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1168/2360] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 05:59:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 05:59:12.628552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56c816-b83d-43d3-8de7-953de39d8ad7 service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 05:59:12.638774 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56c816-b83d-43d3-8de7-953de39d8ad7 service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 05:59:12.643286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e25c2-920b-4553-929e-72f8800060cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 05:59:12.643887 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e25c2-920b-4553-929e-72f8800060cd service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 05:59:12.644195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e25c2-920b-4553-929e-72f8800060cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 05:59:12.644465 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e25c2-920b-4553-929e-72f8800060cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 05:59:12.646722 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dd8e25c2-920b-4553-929e-72f8800060cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 05:59:12.656716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e25c2-920b-4553-929e-72f8800060cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 05:59:12.658134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e25c2-920b-4553-929e-72f8800060cd service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 05:59:12.666276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1193/2361] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 05:59:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 05:59:22.300577 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=84 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 05:59:22.662882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=70 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:00:47.888030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feacfe3-1735-4f2a-b710-7d0c86c3d021 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:00:47.918591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feacfe3-1735-4f2a-b710-7d0c86c3d021 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:00:47.922483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcacf603-33d7-41f3-b1a0-075aa4312ba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:00:47.923088 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacf603-33d7-41f3-b1a0-075aa4312ba1 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:00:47.923368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacf603-33d7-41f3-b1a0-075aa4312ba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:00:47.923640 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcacf603-33d7-41f3-b1a0-075aa4312ba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:00:47.931934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcacf603-33d7-41f3-b1a0-075aa4312ba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:00:47.933396 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcacf603-33d7-41f3-b1a0-075aa4312ba1 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:00:47.961780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1169/2362] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:00:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:00:57.960038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=22 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:01:04.525385 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e25c2-920b-4553-929e-72f8800060cd service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:01:04.552807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e25c2-920b-4553-929e-72f8800060cd service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:01:04.556619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb527f16-6079-4594-9e49-880e67c7c942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:01:04.557274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb527f16-6079-4594-9e49-880e67c7c942 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:01:04.557598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb527f16-6079-4594-9e49-880e67c7c942 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:01:04.557912 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb527f16-6079-4594-9e49-880e67c7c942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:01:04.570363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb527f16-6079-4594-9e49-880e67c7c942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:01:04.571806 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb527f16-6079-4594-9e49-880e67c7c942 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:01:04.580524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1194/2363] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:01:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:01:07.308260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3755e-2fa4-46f5-ad9c-d7710c3c2ba8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:01:07.308744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3755e-2fa4-46f5-ad9c-d7710c3c2ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:01:07.308832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3755e-2fa4-46f5-ad9c-d7710c3c2ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:01:07.345748 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0dd3755e-2fa4-46f5-ad9c-d7710c3c2ba8 None None] MFA Rules not processed for user `32df696da09746748786b94aa8b6721d`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:01:07.346951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0dd3755e-2fa4-46f5-ad9c-d7710c3c2ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:01:07.353644 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1170/2364] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:01:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:01:14.576107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=15 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:01:17.343406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=10 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:01:23.245938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb527f16-6079-4594-9e49-880e67c7c942 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:01:23.259326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb527f16-6079-4594-9e49-880e67c7c942 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:01:23.265278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99a42f-b045-47a4-8db8-ed1e4d74cf2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:01:23.265894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99a42f-b045-47a4-8db8-ed1e4d74cf2c service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:01:23.266186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99a42f-b045-47a4-8db8-ed1e4d74cf2c service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:01:23.266465 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99a42f-b045-47a4-8db8-ed1e4d74cf2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:01:23.276725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99a42f-b045-47a4-8db8-ed1e4d74cf2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:01:23.278333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99a42f-b045-47a4-8db8-ed1e4d74cf2c service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:01:23.290573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1195/2365] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:01:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:01:33.285919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=6 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:01:47.288377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3755e-2fa4-46f5-ad9c-d7710c3c2ba8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:01:47.318343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3755e-2fa4-46f5-ad9c-d7710c3c2ba8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:01:47.323682 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e8972-6544-477d-8824-3638d87fb999 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:01:47.324299 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e8972-6544-477d-8824-3638d87fb999 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:01:47.324576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e8972-6544-477d-8824-3638d87fb999 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:01:47.324844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e8972-6544-477d-8824-3638d87fb999 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:01:47.333464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e8972-6544-477d-8824-3638d87fb999 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:01:47.335058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e8972-6544-477d-8824-3638d87fb999 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:01:47.361746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1171/2366] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:01:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:01:48.583560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99a42f-b045-47a4-8db8-ed1e4d74cf2c service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:01:48.592694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99a42f-b045-47a4-8db8-ed1e4d74cf2c service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:01:48.596911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90901c5-f11f-4412-a1f4-8cd5c4495fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:01:48.597502 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90901c5-f11f-4412-a1f4-8cd5c4495fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:01:48.597783 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90901c5-f11f-4412-a1f4-8cd5c4495fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:01:48.598058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90901c5-f11f-4412-a1f4-8cd5c4495fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:01:48.608017 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90901c5-f11f-4412-a1f4-8cd5c4495fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:01:48.610106 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90901c5-f11f-4412-a1f4-8cd5c4495fc4 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:01:48.620812 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1196/2367] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:01:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:01:57.359711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=20 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:01:58.616067 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=6 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:02:38.595807 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e8972-6544-477d-8824-3638d87fb999 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:02:38.606989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e8972-6544-477d-8824-3638d87fb999 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:02:38.613398 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222894e-0e11-4851-965b-4bdb7324c19f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:02:38.613995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222894e-0e11-4851-965b-4bdb7324c19f service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:02:38.614401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222894e-0e11-4851-965b-4bdb7324c19f service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:02:38.614673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222894e-0e11-4851-965b-4bdb7324c19f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:02:38.622836 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222894e-0e11-4851-965b-4bdb7324c19f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:02:38.625259 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222894e-0e11-4851-965b-4bdb7324c19f service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:02:38.653465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1172/2368] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:02:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:02:48.239312 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90901c5-f11f-4412-a1f4-8cd5c4495fc4 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:02:48.248301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90901c5-f11f-4412-a1f4-8cd5c4495fc4 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:02:48.252391 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a634d-b93b-404e-b1af-c43426b4054e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:02:48.252976 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a634d-b93b-404e-b1af-c43426b4054e service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:02:48.253250 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a634d-b93b-404e-b1af-c43426b4054e service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:02:48.253520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a634d-b93b-404e-b1af-c43426b4054e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:02:48.261950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a634d-b93b-404e-b1af-c43426b4054e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:02:48.263380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a634d-b93b-404e-b1af-c43426b4054e service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:02:48.290286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1197/2369] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:02:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:02:48.650862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=13 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:02:58.287717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=13 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:03:06.234706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222894e-0e11-4851-965b-4bdb7324c19f service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:03:06.243919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222894e-0e11-4851-965b-4bdb7324c19f service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:03:06.250017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d586b6-dced-406a-81dd-c8761d0892ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:03:06.250959 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d586b6-dced-406a-81dd-c8761d0892ae service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:03:06.251342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d586b6-dced-406a-81dd-c8761d0892ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:03:06.251617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d586b6-dced-406a-81dd-c8761d0892ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:03:06.261836 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d586b6-dced-406a-81dd-c8761d0892ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:03:06.263255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d586b6-dced-406a-81dd-c8761d0892ae service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:03:06.274195 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1173/2370] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:03:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:03:07.914929 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a634d-b93b-404e-b1af-c43426b4054e service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:03:07.916410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8e1a634d-b93b-404e-b1af-c43426b4054e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:03:07.941310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a634d-b93b-404e-b1af-c43426b4054e service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:03:07.944895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84effd21-fd7f-4ee7-9bfd-5e82dc34c7f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:03:07.945486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84effd21-fd7f-4ee7-9bfd-5e82dc34c7f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:03:07.945754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84effd21-fd7f-4ee7-9bfd-5e82dc34c7f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:03:07.946024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84effd21-fd7f-4ee7-9bfd-5e82dc34c7f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:03:07.954267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84effd21-fd7f-4ee7-9bfd-5e82dc34c7f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:03:07.955672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84effd21-fd7f-4ee7-9bfd-5e82dc34c7f1 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:03:07.981707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1198/2371] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:03:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:03:16.269243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=6 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:03:17.979131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=18 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:04:12.762106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d586b6-dced-406a-81dd-c8761d0892ae service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:04:12.763589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-28d586b6-dced-406a-81dd-c8761d0892ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:04:12.848800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d586b6-dced-406a-81dd-c8761d0892ae service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:04:12.854354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59794753-a915-4e19-b101-fcb7030df349 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:04:12.854957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59794753-a915-4e19-b101-fcb7030df349 service placement] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:04:12.855233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59794753-a915-4e19-b101-fcb7030df349 service placement] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:04:12.855500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59794753-a915-4e19-b101-fcb7030df349 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:04:12.858228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-59794753-a915-4e19-b101-fcb7030df349 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:04:12.869904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59794753-a915-4e19-b101-fcb7030df349 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:04:12.871493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59794753-a915-4e19-b101-fcb7030df349 service placement] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:04:12.881477 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1174/2372] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:04:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:04:22.877260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=37 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:06:07.413817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84effd21-fd7f-4ee7-9bfd-5e82dc34c7f1 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:06:07.468433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84effd21-fd7f-4ee7-9bfd-5e82dc34c7f1 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:06:07.472087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8870373-969b-4190-a47a-900dd8e4d879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:06:07.472626 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8870373-969b-4190-a47a-900dd8e4d879 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:06:07.472906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8870373-969b-4190-a47a-900dd8e4d879 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:06:07.473178 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8870373-969b-4190-a47a-900dd8e4d879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:06:07.475012 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f8870373-969b-4190-a47a-900dd8e4d879 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:06:07.498286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8870373-969b-4190-a47a-900dd8e4d879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:06:07.499914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8870373-969b-4190-a47a-900dd8e4d879 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:06:07.508681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1199/2373] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:06:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:06:17.504682 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=31 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:06:47.550817 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59794753-a915-4e19-b101-fcb7030df349 service placement] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:06:47.552630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-59794753-a915-4e19-b101-fcb7030df349 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:06:47.619063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59794753-a915-4e19-b101-fcb7030df349 service placement] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:06:47.622639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20faed48-d200-401a-a5f5-930bfc180226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:06:47.623248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faed48-d200-401a-a5f5-930bfc180226 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:06:47.623529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faed48-d200-401a-a5f5-930bfc180226 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:06:47.623804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faed48-d200-401a-a5f5-930bfc180226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:06:47.625825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-20faed48-d200-401a-a5f5-930bfc180226 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:06:47.651539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8870373-969b-4190-a47a-900dd8e4d879 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:06:47.661585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8870373-969b-4190-a47a-900dd8e4d879 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:06:47.665370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0c1ca-4490-4654-9b0b-5d044262d41c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:06:47.665965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0c1ca-4490-4654-9b0b-5d044262d41c service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:06:47.666248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0c1ca-4490-4654-9b0b-5d044262d41c service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:06:47.666541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0c1ca-4490-4654-9b0b-5d044262d41c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:06:47.668440 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-20c0c1ca-4490-4654-9b0b-5d044262d41c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:06:47.705115 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20faed48-d200-401a-a5f5-930bfc180226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:06:47.706944 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20faed48-d200-401a-a5f5-930bfc180226 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:06:47.717034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1175/2374] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:06:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:06:47.748471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0c1ca-4490-4654-9b0b-5d044262d41c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:06:47.749913 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0c1ca-4490-4654-9b0b-5d044262d41c service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:06:47.759894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1200/2375] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:06:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:06:48.219280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20faed48-d200-401a-a5f5-930bfc180226 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:06:48.229693 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20faed48-d200-401a-a5f5-930bfc180226 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:06:48.233775 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0434b5-1330-4d63-a3a1-c5b18319cc07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:06:48.234381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0434b5-1330-4d63-a3a1-c5b18319cc07 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:06:48.235296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0434b5-1330-4d63-a3a1-c5b18319cc07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:06:48.235626 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0434b5-1330-4d63-a3a1-c5b18319cc07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:06:48.238491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-de0434b5-1330-4d63-a3a1-c5b18319cc07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:06:48.354723 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0434b5-1330-4d63-a3a1-c5b18319cc07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:06:48.356777 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0434b5-1330-4d63-a3a1-c5b18319cc07 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:06:48.370275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1176/2376] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:06:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:06:48.858484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0c1ca-4490-4654-9b0b-5d044262d41c service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:06:48.874214 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0c1ca-4490-4654-9b0b-5d044262d41c service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:06:48.887209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76e5f6-9f90-4818-942a-3bd5be23d414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:06:48.889639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76e5f6-9f90-4818-942a-3bd5be23d414 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:06:48.890208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76e5f6-9f90-4818-942a-3bd5be23d414 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:06:48.890871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76e5f6-9f90-4818-942a-3bd5be23d414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:06:48.893585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9e76e5f6-9f90-4818-942a-3bd5be23d414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:06:49.015094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76e5f6-9f90-4818-942a-3bd5be23d414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:06:49.017146 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76e5f6-9f90-4818-942a-3bd5be23d414 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:06:49.038742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1201/2377] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:06:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:06:51.768920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0434b5-1330-4d63-a3a1-c5b18319cc07 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:06:51.775621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76e5f6-9f90-4818-942a-3bd5be23d414 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:06:51.778033 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9e76e5f6-9f90-4818-942a-3bd5be23d414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:06:51.781501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0434b5-1330-4d63-a3a1-c5b18319cc07 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:06:51.786073 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2808a-0bda-4875-ae83-49b924234059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:06:51.786752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2808a-0bda-4875-ae83-49b924234059 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:06:51.787094 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2808a-0bda-4875-ae83-49b924234059 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:06:51.787434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2808a-0bda-4875-ae83-49b924234059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:06:51.791215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a6c2808a-0bda-4875-ae83-49b924234059 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:06:51.803803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76e5f6-9f90-4818-942a-3bd5be23d414 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:06:51.830828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:06:51.851522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02d19c-c0c3-4c61-b29b-e512a651755e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:06:51.852314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02d19c-c0c3-4c61-b29b-e512a651755e service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:06:51.852614 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02d19c-c0c3-4c61-b29b-e512a651755e service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:06:51.852881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02d19c-c0c3-4c61-b29b-e512a651755e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:06:51.865364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6a02d19c-c0c3-4c61-b29b-e512a651755e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:06:51.927029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02d19c-c0c3-4c61-b29b-e512a651755e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:06:51.931394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02d19c-c0c3-4c61-b29b-e512a651755e service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:06:51.951249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2808a-0bda-4875-ae83-49b924234059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:06:51.954534 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2808a-0bda-4875-ae83-49b924234059 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:06:51.960672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1202/2378] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:06:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:06:52.019391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1177/2379] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:06:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:06:52.854424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02d19c-c0c3-4c61-b29b-e512a651755e service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:06:52.855991 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6a02d19c-c0c3-4c61-b29b-e512a651755e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:06:52.869644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02d19c-c0c3-4c61-b29b-e512a651755e service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:06:52.875326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7904f9c-d9fd-43ec-a5f4-0f25ca52ccac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:06:52.876579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7904f9c-d9fd-43ec-a5f4-0f25ca52ccac service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:06:52.877467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7904f9c-d9fd-43ec-a5f4-0f25ca52ccac service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:06:52.877821 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7904f9c-d9fd-43ec-a5f4-0f25ca52ccac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:06:52.879731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f7904f9c-d9fd-43ec-a5f4-0f25ca52ccac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:06:52.892322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7904f9c-d9fd-43ec-a5f4-0f25ca52ccac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:06:52.894516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7904f9c-d9fd-43ec-a5f4-0f25ca52ccac service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:06:52.907387 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1203/2380] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:06:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:07:02.009504 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=11 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:02.902901 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=86 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:04.289028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2808a-0bda-4875-ae83-49b924234059 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:04.303465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2808a-0bda-4875-ae83-49b924234059 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:04.309340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79b0da-1875-4ab3-b426-aa0a09672440 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:04.310140 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79b0da-1875-4ab3-b426-aa0a09672440 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:04.310583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79b0da-1875-4ab3-b426-aa0a09672440 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:04.311166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79b0da-1875-4ab3-b426-aa0a09672440 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:04.321071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0b79b0da-1875-4ab3-b426-aa0a09672440 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:04.334918 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79b0da-1875-4ab3-b426-aa0a09672440 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:04.336772 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79b0da-1875-4ab3-b426-aa0a09672440 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:04.349977 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1178/2381] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:07:07.385959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66f034-110d-4fe7-ba68-56836698d4b2 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:07.386480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66f034-110d-4fe7-ba68-56836698d4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:07.386480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66f034-110d-4fe7-ba68-56836698d4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:07.407077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8d66f034-110d-4fe7-ba68-56836698d4b2 None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:07.408978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8d66f034-110d-4fe7-ba68-56836698d4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:07.417953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1204/2382] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 06:07:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:07:07.478340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79b0da-1875-4ab3-b426-aa0a09672440 service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:07.492351 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79b0da-1875-4ab3-b426-aa0a09672440 service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:07.499111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b7976-cc29-472b-ac0c-733353d19a02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:07.499672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b7976-cc29-472b-ac0c-733353d19a02 service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:07.499955 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b7976-cc29-472b-ac0c-733353d19a02 service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:07.500300 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b7976-cc29-472b-ac0c-733353d19a02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:07.510079 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b7976-cc29-472b-ac0c-733353d19a02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:07.511680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b7976-cc29-472b-ac0c-733353d19a02 service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:07.521694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1179/2383] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 06:07:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 06:07:17.401384 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:17.493900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49a479-4519-4eec-bd8f-8ab692881c54 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:17.494457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49a479-4519-4eec-bd8f-8ab692881c54 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:17.494908 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49a479-4519-4eec-bd8f-8ab692881c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:17.520435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=12 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:17.548314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-df49a479-4519-4eec-bd8f-8ab692881c54 None None] MFA Rules not processed for user `54b72a430d9d4d43954eb56d703079b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:17.550789 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-df49a479-4519-4eec-bd8f-8ab692881c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:17.624772 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1205/2384] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:18.040490 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b7976-cc29-472b-ac0c-733353d19a02 service swift] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:18.042356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-af3b7976-cc29-472b-ac0c-733353d19a02 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:18.057505 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b7976-cc29-472b-ac0c-733353d19a02 service swift] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:18.061553 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a67d5b-cf99-4464-929b-52a4271e8eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:18.062674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a67d5b-cf99-4464-929b-52a4271e8eeb service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:18.063172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a67d5b-cf99-4464-929b-52a4271e8eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:18.063826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a67d5b-cf99-4464-929b-52a4271e8eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:18.066345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-37a67d5b-cf99-4464-929b-52a4271e8eeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:18.078273 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a67d5b-cf99-4464-929b-52a4271e8eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:18.080638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a67d5b-cf99-4464-929b-52a4271e8eeb service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:18.097173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1180/2385] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:07:18.602638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49a479-4519-4eec-bd8f-8ab692881c54 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:18.605096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-df49a479-4519-4eec-bd8f-8ab692881c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:18.641654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49a479-4519-4eec-bd8f-8ab692881c54 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:18.653996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc086b6-6d02-432d-817e-7e0b6c73cd02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:18.654995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc086b6-6d02-432d-817e-7e0b6c73cd02 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:18.655695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc086b6-6d02-432d-817e-7e0b6c73cd02 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:18.656631 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc086b6-6d02-432d-817e-7e0b6c73cd02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:18.660715 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-fbc086b6-6d02-432d-817e-7e0b6c73cd02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:18.675246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc086b6-6d02-432d-817e-7e0b6c73cd02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:18.677019 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc086b6-6d02-432d-817e-7e0b6c73cd02 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:18.704431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1206/2386] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:07:28.090940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=6 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:28.503459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2be18-7ce6-4df6-8426-da3e63e4dc44 None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:28.504163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2be18-7ce6-4df6-8426-da3e63e4dc44 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:28.504630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2be18-7ce6-4df6-8426-da3e63e4dc44 None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:28.506516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1181/2387] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 06:07:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 06:07:28.518075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f966f5-ff21-4a6b-b9d9-2c599494d302 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:28.518960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f966f5-ff21-4a6b-b9d9-2c599494d302 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:28.519411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f966f5-ff21-4a6b-b9d9-2c599494d302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:28.590921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-48f966f5-ff21-4a6b-b9d9-2c599494d302 None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:28.592607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-48f966f5-ff21-4a6b-b9d9-2c599494d302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:28.625188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1207/2388] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 06:07:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:28.641332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2be18-7ce6-4df6-8426-da3e63e4dc44 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:28.644224 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ffd2be18-7ce6-4df6-8426-da3e63e4dc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:28.778880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2be18-7ce6-4df6-8426-da3e63e4dc44 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:28.783794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe0efa-246c-415b-bfca-3797f229614a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0330c79c391d463b91f99e030c6dcc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:28.784725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe0efa-246c-415b-bfca-3797f229614a service glance] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:28.785103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe0efa-246c-415b-bfca-3797f229614a service glance] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:28.786036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe0efa-246c-415b-bfca-3797f229614a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:28.797350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe0efa-246c-415b-bfca-3797f229614a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:28.799672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe0efa-246c-415b-bfca-3797f229614a service glance] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:28.813495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1182/2389] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:07:28.889488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9e654-56a8-413f-81da-548d0d8c6f05 None None] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:28.889775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9e654-56a8-413f-81da-548d0d8c6f05 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:28.890238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9e654-56a8-413f-81da-548d0d8c6f05 None None] PATH_INFO: `/` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:28.891819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1208/2390] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 06:07:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 06:07:28.900301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce9c3b-1f90-47c6-9555-10d63a085dfb None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:28.900641 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce9c3b-1f90-47c6-9555-10d63a085dfb None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:28.900917 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce9c3b-1f90-47c6-9555-10d63a085dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:28.944514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d2ce9c3b-1f90-47c6-9555-10d63a085dfb None None] MFA Rules not processed for user `0bf2a5de0e744505a1317ce9fa224643`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:28.945794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d2ce9c3b-1f90-47c6-9555-10d63a085dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:28.953444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1183/2391] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 06:07:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:28.966403 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9e654-56a8-413f-81da-548d0d8c6f05 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:28.978810 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9e654-56a8-413f-81da-548d0d8c6f05 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:28.983444 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002dec7-0012-436b-87b4-701f7e430320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:28.984193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002dec7-0012-436b-87b4-701f7e430320 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:28.984570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002dec7-0012-436b-87b4-701f7e430320 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:28.984940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002dec7-0012-436b-87b4-701f7e430320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:28.993859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002dec7-0012-436b-87b4-701f7e430320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:28.995637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002dec7-0012-436b-87b4-701f7e430320 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:29.008527 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1209/2392] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:07:29.137344 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325cbcc-2a43-4535-98bd-c91ade3eaaca None None] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:29.137873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325cbcc-2a43-4535-98bd-c91ade3eaaca None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:29.138290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325cbcc-2a43-4535-98bd-c91ade3eaaca None None] PATH_INFO: `/` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:29.139844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1184/2393] 10.222.0.33 () {54 vars in 921 bytes} [Sun Sep 11 06:07:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 06:07:29.146848 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d95b79-e277-40e2-add4-2ec824d9bb14 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:29.147521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d95b79-e277-40e2-add4-2ec824d9bb14 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:29.149965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d95b79-e277-40e2-add4-2ec824d9bb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:29.177755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a0d95b79-e277-40e2-add4-2ec824d9bb14 None None] MFA Rules not processed for user `3bd5a9a09f034c05815c631a44d1cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:29.181223 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a0d95b79-e277-40e2-add4-2ec824d9bb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:29.201768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1210/2394] 10.222.0.33 () {58 vars in 1018 bytes} [Sun Sep 11 06:07:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:29.211701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325cbcc-2a43-4535-98bd-c91ade3eaaca None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:29.230164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325cbcc-2a43-4535-98bd-c91ade3eaaca None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:29.234562 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b2110-bd22-4b24-bff2-7814214d3f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:29.235459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b2110-bd22-4b24-bff2-7814214d3f25 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:29.236975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b2110-bd22-4b24-bff2-7814214d3f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:29.237909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b2110-bd22-4b24-bff2-7814214d3f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:29.250740 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b2110-bd22-4b24-bff2-7814214d3f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:29.253661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b2110-bd22-4b24-bff2-7814214d3f25 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:29.271410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1185/2395] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:07:33.640341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb4b59-7b35-4143-8ad6-1e5f85a5fe21 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:33.640789 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb4b59-7b35-4143-8ad6-1e5f85a5fe21 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:33.640887 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb4b59-7b35-4143-8ad6-1e5f85a5fe21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:33.661491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c2cb4b59-7b35-4143-8ad6-1e5f85a5fe21 None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:33.662701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c2cb4b59-7b35-4143-8ad6-1e5f85a5fe21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:33.669850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1211/2396] 10.222.0.33 () {58 vars in 1009 bytes} [Sun Sep 11 06:07:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:33.684217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b2110-bd22-4b24-bff2-7814214d3f25 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:33.696535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b2110-bd22-4b24-bff2-7814214d3f25 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:33.701648 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3a272-3562-41c0-8a9e-c3da1c9c7a9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:33.702339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3a272-3562-41c0-8a9e-c3da1c9c7a9a service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:33.702711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3a272-3562-41c0-8a9e-c3da1c9c7a9a service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:33.703040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3a272-3562-41c0-8a9e-c3da1c9c7a9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:33.711454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa3a272-3562-41c0-8a9e-c3da1c9c7a9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:33.713142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa3a272-3562-41c0-8a9e-c3da1c9c7a9a service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:33.721862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1186/2397] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 06:07:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 06:07:37.571328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb4b59-7b35-4143-8ad6-1e5f85a5fe21 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:37.616882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb4b59-7b35-4143-8ad6-1e5f85a5fe21 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:37.622725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4ebdb-fdab-4b72-ab8a-3129774c74b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:37.623352 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4ebdb-fdab-4b72-ab8a-3129774c74b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:37.623633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4ebdb-fdab-4b72-ab8a-3129774c74b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:37.623902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4ebdb-fdab-4b72-ab8a-3129774c74b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:37.628882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d6b4ebdb-fdab-4b72-ab8a-3129774c74b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:37.658913 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4ebdb-fdab-4b72-ab8a-3129774c74b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:37.660665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4ebdb-fdab-4b72-ab8a-3129774c74b3 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:37.694955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1212/2398] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:07:42.752584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3a272-3562-41c0-8a9e-c3da1c9c7a9a service swift] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:42.774394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3a272-3562-41c0-8a9e-c3da1c9c7a9a service swift] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:42.785217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa131e-f0e7-4b3d-b0cc-c254a17dd40e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:42.786139 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa131e-f0e7-4b3d-b0cc-c254a17dd40e service placement] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:42.786621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa131e-f0e7-4b3d-b0cc-c254a17dd40e service placement] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:42.787079 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa131e-f0e7-4b3d-b0cc-c254a17dd40e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:42.794746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f2aa131e-f0e7-4b3d-b0cc-c254a17dd40e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:42.808798 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa131e-f0e7-4b3d-b0cc-c254a17dd40e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:42.810667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa131e-f0e7-4b3d-b0cc-c254a17dd40e service placement] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:42.823541 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1187/2399] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:07:42.871063 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4ebdb-fdab-4b72-ab8a-3129774c74b3 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:42.872790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d6b4ebdb-fdab-4b72-ab8a-3129774c74b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:42.900450 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4ebdb-fdab-4b72-ab8a-3129774c74b3 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:42.905650 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee93b5-949e-44ef-a425-8d81af847cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:42.906278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee93b5-949e-44ef-a425-8d81af847cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:42.906553 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee93b5-949e-44ef-a425-8d81af847cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:42.906824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee93b5-949e-44ef-a425-8d81af847cb3 admin admin] PATH_INFO: `/v3/users/32df696da09746748786b94aa8b6721d` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:42.923281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee93b5-949e-44ef-a425-8d81af847cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32df696da09746748786b94aa8b6721d)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:42.925510 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee93b5-949e-44ef-a425-8d81af847cb3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:42.976840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9aee93b5-949e-44ef-a425-8d81af847cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32df696da09746748786b94aa8b6721d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:42.977497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9aee93b5-949e-44ef-a425-8d81af847cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32df696da09746748786b94aa8b6721d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:42.984717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9aee93b5-949e-44ef-a425-8d81af847cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32df696da09746748786b94aa8b6721d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:42.986907 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:42.994511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9aee93b5-949e-44ef-a425-8d81af847cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32df696da09746748786b94aa8b6721d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:43.008500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9aee93b5-949e-44ef-a425-8d81af847cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32df696da09746748786b94aa8b6721d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:43.014434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1213/2400] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:07:42 2022] DELETE /identity/v3/users/32df696da09746748786b94aa8b6721d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:43.223790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa131e-f0e7-4b3d-b0cc-c254a17dd40e service placement] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:43.239923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa131e-f0e7-4b3d-b0cc-c254a17dd40e service placement] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:43.303093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6645df5-167a-47c3-8792-e3c81aa5999f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:43.303728 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6645df5-167a-47c3-8792-e3c81aa5999f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:43.304032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6645df5-167a-47c3-8792-e3c81aa5999f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:43.304315 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6645df5-167a-47c3-8792-e3c81aa5999f admin admin] PATH_INFO: `/v3/projects/d47e270753bc4fe6ac5ba1f2b0d5d2af` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:43.305461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6645df5-167a-47c3-8792-e3c81aa5999f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d47e270753bc4fe6ac5ba1f2b0d5d2af)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:43.307482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6645df5-167a-47c3-8792-e3c81aa5999f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:43.342006 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:43.361221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b6645df5-167a-47c3-8792-e3c81aa5999f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd47e270753bc4fe6ac5ba1f2b0d5d2af', 'request_id': 'req-b6645df5-167a-47c3-8792-e3c81aa5999f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:43.369465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b6645df5-167a-47c3-8792-e3c81aa5999f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd47e270753bc4fe6ac5ba1f2b0d5d2af', 'request_id': 'req-b6645df5-167a-47c3-8792-e3c81aa5999f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:43.378146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b6645df5-167a-47c3-8792-e3c81aa5999f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd47e270753bc4fe6ac5ba1f2b0d5d2af', 'request_id': 'req-b6645df5-167a-47c3-8792-e3c81aa5999f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:43.388445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b6645df5-167a-47c3-8792-e3c81aa5999f admin admin] The token cache is being invalidate because project d47e270753bc4fe6ac5ba1f2b0d5d2af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:07:43.389306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b6645df5-167a-47c3-8792-e3c81aa5999f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:43.389967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b6645df5-167a-47c3-8792-e3c81aa5999f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:43.391461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1188/2401] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:07:43 2022] DELETE /identity/v3/projects/d47e270753bc4fe6ac5ba1f2b0d5d2af => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:43.396621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117390a-ecb3-4db7-85fd-9de07648cab3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:43.396912 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117390a-ecb3-4db7-85fd-9de07648cab3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:43.397188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117390a-ecb3-4db7-85fd-9de07648cab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:43.448446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a117390a-ecb3-4db7-85fd-9de07648cab3 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:43.449656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a117390a-ecb3-4db7-85fd-9de07648cab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:43.508886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1214/2402] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:43.517004 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dbe68-df3f-4791-998f-ac29696d828d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:43.517496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dbe68-df3f-4791-998f-ac29696d828d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:43.517799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dbe68-df3f-4791-998f-ac29696d828d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:43.538164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ee2dbe68-df3f-4791-998f-ac29696d828d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:43.539382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ee2dbe68-df3f-4791-998f-ac29696d828d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:43.548694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1189/2403] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:43.552896 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117390a-ecb3-4db7-85fd-9de07648cab3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:43.562940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117390a-ecb3-4db7-85fd-9de07648cab3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:43.567803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2c7b8-954b-4657-af19-ea313b198294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:43.568712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2c7b8-954b-4657-af19-ea313b198294 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:43.568994 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2c7b8-954b-4657-af19-ea313b198294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:43.569312 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2c7b8-954b-4657-af19-ea313b198294 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:43.569972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2c7b8-954b-4657-af19-ea313b198294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:07:43.570275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2c7b8-954b-4657-af19-ea313b198294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:43.571914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2c7b8-954b-4657-af19-ea313b198294 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:43.584655 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1215/2404] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:07:43.589051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2dbe68-df3f-4791-998f-ac29696d828d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:43.599882 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2dbe68-df3f-4791-998f-ac29696d828d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:43.605194 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9327d-a316-4e66-b2b2-88e32857028b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:43.605927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9327d-a316-4e66-b2b2-88e32857028b admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:43.606292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9327d-a316-4e66-b2b2-88e32857028b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:43.606634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9327d-a316-4e66-b2b2-88e32857028b admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:43.607404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9327d-a316-4e66-b2b2-88e32857028b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:43.609572 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9327d-a316-4e66-b2b2-88e32857028b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:43.658296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1190/2405] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:07:43 2022] POST /identity/v3/projects => generated 365 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:07:43.663111 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2c7b8-954b-4657-af19-ea313b198294 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:43.675035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2c7b8-954b-4657-af19-ea313b198294 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:43.760925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fc885-7b79-43e7-9759-1b14dc885c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:43.761395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fc885-7b79-43e7-9759-1b14dc885c3e admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:43.761673 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fc885-7b79-43e7-9759-1b14dc885c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:43.762010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fc885-7b79-43e7-9759-1b14dc885c3e admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:43.762852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fc885-7b79-43e7-9759-1b14dc885c3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:43.764651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fc885-7b79-43e7-9759-1b14dc885c3e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:43.798454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1216/2406] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:07:43 2022] POST /identity/v3/users => generated 283 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:07:43.802636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9327d-a316-4e66-b2b2-88e32857028b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:43.815606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9327d-a316-4e66-b2b2-88e32857028b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:43.820948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea99fd9-6847-4f00-86c1-5b72778d4f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:43.821766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99fd9-6847-4f00-86c1-5b72778d4f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:43.822125 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99fd9-6847-4f00-86c1-5b72778d4f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:43.822476 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99fd9-6847-4f00-86c1-5b72778d4f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:43.823210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea99fd9-6847-4f00-86c1-5b72778d4f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:43.824918 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea99fd9-6847-4f00-86c1-5b72778d4f91 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:43.839733 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1191/2407] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:07:43.840951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fc885-7b79-43e7-9759-1b14dc885c3e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:43.851581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fc885-7b79-43e7-9759-1b14dc885c3e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:43.856246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835e132-d304-4b3d-bf52-e3ce05d05a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:43.856841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835e132-d304-4b3d-bf52-e3ce05d05a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:43.857109 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835e132-d304-4b3d-bf52-e3ce05d05a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:43.857378 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835e132-d304-4b3d-bf52-e3ce05d05a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:43.858014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d835e132-d304-4b3d-bf52-e3ce05d05a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:43.859594 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d835e132-d304-4b3d-bf52-e3ce05d05a52 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:43.870787 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1217/2408] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:07:43.874931 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea99fd9-6847-4f00-86c1-5b72778d4f91 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:43.885646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea99fd9-6847-4f00-86c1-5b72778d4f91 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:43.890397 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f99f3-a091-4968-8b20-13c1f32a27f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:43.891046 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f99f3-a091-4968-8b20-13c1f32a27f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:43.891329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f99f3-a091-4968-8b20-13c1f32a27f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:43.892350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f99f3-a091-4968-8b20-13c1f32a27f6 admin admin] PATH_INFO: `/v3/projects/251cdc2277134ffcad3ccf280f4f03a8/users/cf1e48c577d24b08a066960c411ea3a4/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:43.907587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f99f3-a091-4968-8b20-13c1f32a27f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=251cdc2277134ffcad3ccf280f4f03a8, user_id=cf1e48c577d24b08a066960c411ea3a4, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:43.910430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f99f3-a091-4968-8b20-13c1f32a27f6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:43.923547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1192/2409] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:07:43 2022] PUT /identity/v3/projects/251cdc2277134ffcad3ccf280f4f03a8/users/cf1e48c577d24b08a066960c411ea3a4/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:07:43.933984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f26d37-4bd5-49c8-b7f8-8a4136a211a3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:43.934454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f26d37-4bd5-49c8-b7f8-8a4136a211a3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:43.934554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f26d37-4bd5-49c8-b7f8-8a4136a211a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:43.964494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-62f26d37-4bd5-49c8-b7f8-8a4136a211a3 None None] MFA Rules not processed for user `cf1e48c577d24b08a066960c411ea3a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:43.965701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-62f26d37-4bd5-49c8-b7f8-8a4136a211a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:43.970873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:44.034517 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1218/2410] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:44.045249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f99f3-a091-4968-8b20-13c1f32a27f6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:44.046919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-387f99f3-a091-4968-8b20-13c1f32a27f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:44.103416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f99f3-a091-4968-8b20-13c1f32a27f6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:44.107079 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a68e4-b07a-4a25-acfe-09f915b6dea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:44.107684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a68e4-b07a-4a25-acfe-09f915b6dea1 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:44.107971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a68e4-b07a-4a25-acfe-09f915b6dea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:44.108281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a68e4-b07a-4a25-acfe-09f915b6dea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:44.116806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a68e4-b07a-4a25-acfe-09f915b6dea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:44.118248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a68e4-b07a-4a25-acfe-09f915b6dea1 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:44.127732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1193/2411] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:07:53.746211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f26d37-4bd5-49c8-b7f8-8a4136a211a3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:53.843036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f26d37-4bd5-49c8-b7f8-8a4136a211a3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:53.847634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:53.848304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:53.848587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:53.848863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] PATH_INFO: `/v3/users/cf1e48c577d24b08a066960c411ea3a4` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:53.849995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1e48c577d24b08a066960c411ea3a4)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:53.851926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:53.899430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1e48c577d24b08a066960c411ea3a4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:53.900181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1e48c577d24b08a066960c411ea3a4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:53.907151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1e48c577d24b08a066960c411ea3a4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:53.914593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1e48c577d24b08a066960c411ea3a4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:53.928030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1e48c577d24b08a066960c411ea3a4'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:53.933971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1219/2412] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:07:53 2022] DELETE /identity/v3/users/cf1e48c577d24b08a066960c411ea3a4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:53.939089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84aa65-2fb6-4bb2-9b99-4d68ffdd2387 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:53.939596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84aa65-2fb6-4bb2-9b99-4d68ffdd2387 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:53.939691 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84aa65-2fb6-4bb2-9b99-4d68ffdd2387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:53.958363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4d84aa65-2fb6-4bb2-9b99-4d68ffdd2387 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:53.959553 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4d84aa65-2fb6-4bb2-9b99-4d68ffdd2387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:54.000293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:54.023135 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1194/2413] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:54.033766 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:54.035184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:54.066386 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:54.099098 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5a33f-dfcb-4dde-b4db-833fff0f8714 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:54.102825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006f0c5-d4ba-4719-96a3-d6fbfeb73ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:54.103437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006f0c5-d4ba-4719-96a3-d6fbfeb73ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:54.103718 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006f0c5-d4ba-4719-96a3-d6fbfeb73ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:54.104008 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006f0c5-d4ba-4719-96a3-d6fbfeb73ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:54.113893 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006f0c5-d4ba-4719-96a3-d6fbfeb73ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:54.115332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006f0c5-d4ba-4719-96a3-d6fbfeb73ab0 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:54.124708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1220/2414] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:07:54.402042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84aa65-2fb6-4bb2-9b99-4d68ffdd2387 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:54.403515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4d84aa65-2fb6-4bb2-9b99-4d68ffdd2387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:54.422686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84aa65-2fb6-4bb2-9b99-4d68ffdd2387 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:54.436058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52648da2-75b2-4b0a-b5a4-465f757c309c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:54.436657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52648da2-75b2-4b0a-b5a4-465f757c309c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:54.437027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52648da2-75b2-4b0a-b5a4-465f757c309c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:54.437357 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52648da2-75b2-4b0a-b5a4-465f757c309c admin admin] PATH_INFO: `/v3/projects/251cdc2277134ffcad3ccf280f4f03a8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:54.439356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52648da2-75b2-4b0a-b5a4-465f757c309c admin admin] RBAC: Authorizing `identity:delete_project(project_id=251cdc2277134ffcad3ccf280f4f03a8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:54.441433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52648da2-75b2-4b0a-b5a4-465f757c309c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:54.532400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-52648da2-75b2-4b0a-b5a4-465f757c309c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '251cdc2277134ffcad3ccf280f4f03a8', 'request_id': 'req-52648da2-75b2-4b0a-b5a4-465f757c309c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:54.544332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-52648da2-75b2-4b0a-b5a4-465f757c309c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '251cdc2277134ffcad3ccf280f4f03a8', 'request_id': 'req-52648da2-75b2-4b0a-b5a4-465f757c309c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:54.553119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-52648da2-75b2-4b0a-b5a4-465f757c309c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '251cdc2277134ffcad3ccf280f4f03a8', 'request_id': 'req-52648da2-75b2-4b0a-b5a4-465f757c309c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:54.572762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-52648da2-75b2-4b0a-b5a4-465f757c309c admin admin] The token cache is being invalidate because project 251cdc2277134ffcad3ccf280f4f03a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:07:54.573233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-52648da2-75b2-4b0a-b5a4-465f757c309c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:54.574041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-52648da2-75b2-4b0a-b5a4-465f757c309c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:54.579678 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1195/2415] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:07:54 2022] DELETE /identity/v3/projects/251cdc2277134ffcad3ccf280f4f03a8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:54.581475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0c985-8b56-42b1-bfa2-c09cf411bb7a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:54.581992 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0c985-8b56-42b1-bfa2-c09cf411bb7a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:54.582472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0c985-8b56-42b1-bfa2-c09cf411bb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:54.606908 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-39a0c985-8b56-42b1-bfa2-c09cf411bb7a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:54.608489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-39a0c985-8b56-42b1-bfa2-c09cf411bb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:54.687588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1221/2416] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:54.755867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318ccad-57d6-49e3-a9b5-e59ccd17ce03 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:54.756089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318ccad-57d6-49e3-a9b5-e59ccd17ce03 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:54.756367 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318ccad-57d6-49e3-a9b5-e59ccd17ce03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:54.777011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f318ccad-57d6-49e3-a9b5-e59ccd17ce03 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:54.778228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f318ccad-57d6-49e3-a9b5-e59ccd17ce03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:54.787023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1196/2417] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:54.791417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0c985-8b56-42b1-bfa2-c09cf411bb7a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:54.803570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0c985-8b56-42b1-bfa2-c09cf411bb7a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:54.815208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21117dc0-71d3-47fa-9849-fa9a42e077de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:54.815895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21117dc0-71d3-47fa-9849-fa9a42e077de admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:54.816263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21117dc0-71d3-47fa-9849-fa9a42e077de admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:54.816536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21117dc0-71d3-47fa-9849-fa9a42e077de admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:54.817194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21117dc0-71d3-47fa-9849-fa9a42e077de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:07:54.817498 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21117dc0-71d3-47fa-9849-fa9a42e077de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:54.819164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21117dc0-71d3-47fa-9849-fa9a42e077de admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:54.833047 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1222/2418] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:07:54.837087 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318ccad-57d6-49e3-a9b5-e59ccd17ce03 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:54.848302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318ccad-57d6-49e3-a9b5-e59ccd17ce03 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:54.856826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efb87c-bba4-4054-9c42-95bd99a3334b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:54.857506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efb87c-bba4-4054-9c42-95bd99a3334b admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:54.857709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efb87c-bba4-4054-9c42-95bd99a3334b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:54.857985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efb87c-bba4-4054-9c42-95bd99a3334b admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:54.858708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efb87c-bba4-4054-9c42-95bd99a3334b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:54.861574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efb87c-bba4-4054-9c42-95bd99a3334b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:54.896854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1197/2419] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:07:54 2022] POST /identity/v3/projects => generated 363 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:07:54.899229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21117dc0-71d3-47fa-9849-fa9a42e077de admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:54.909927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21117dc0-71d3-47fa-9849-fa9a42e077de admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:54.975558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69943cca-6ee8-4dcd-9a5c-590c99f344b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:54.976343 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69943cca-6ee8-4dcd-9a5c-590c99f344b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:54.976741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69943cca-6ee8-4dcd-9a5c-590c99f344b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:54.977108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69943cca-6ee8-4dcd-9a5c-590c99f344b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:54.977899 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69943cca-6ee8-4dcd-9a5c-590c99f344b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:54.980127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69943cca-6ee8-4dcd-9a5c-590c99f344b4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:55.005383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1223/2420] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:07:54 2022] POST /identity/v3/users => generated 282 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:07:55.009519 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efb87c-bba4-4054-9c42-95bd99a3334b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:55.020360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efb87c-bba4-4054-9c42-95bd99a3334b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:55.025476 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07008c9f-6396-4bb7-b9d7-89bb963021ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:55.026080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07008c9f-6396-4bb7-b9d7-89bb963021ed admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:55.026360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07008c9f-6396-4bb7-b9d7-89bb963021ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:55.026633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07008c9f-6396-4bb7-b9d7-89bb963021ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:55.027279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07008c9f-6396-4bb7-b9d7-89bb963021ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:55.028913 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07008c9f-6396-4bb7-b9d7-89bb963021ed admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:55.040293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1198/2421] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:07:55.044129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69943cca-6ee8-4dcd-9a5c-590c99f344b4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:55.055684 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69943cca-6ee8-4dcd-9a5c-590c99f344b4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:55.060427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57967b-b4d3-4e7e-a2d4-12215886a502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:55.061028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57967b-b4d3-4e7e-a2d4-12215886a502 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:55.061310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57967b-b4d3-4e7e-a2d4-12215886a502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:55.061582 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57967b-b4d3-4e7e-a2d4-12215886a502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:55.062232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57967b-b4d3-4e7e-a2d4-12215886a502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:55.064711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57967b-b4d3-4e7e-a2d4-12215886a502 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:55.075556 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1224/2422] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:07:55.079737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07008c9f-6396-4bb7-b9d7-89bb963021ed admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:55.090139 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07008c9f-6396-4bb7-b9d7-89bb963021ed admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:55.095000 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a271-6d36-4558-8de9-c8ce1c57bd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:55.095659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a271-6d36-4558-8de9-c8ce1c57bd87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:55.095940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a271-6d36-4558-8de9-c8ce1c57bd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:55.096246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a271-6d36-4558-8de9-c8ce1c57bd87 admin admin] PATH_INFO: `/v3/projects/0b47cdb6680246c7a93abd43b2b3641d/users/3d6f15ddbf1049eda9370e40bb31f857/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:55.111700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11a271-6d36-4558-8de9-c8ce1c57bd87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b47cdb6680246c7a93abd43b2b3641d, user_id=3d6f15ddbf1049eda9370e40bb31f857, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:55.114498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11a271-6d36-4558-8de9-c8ce1c57bd87 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:55.128333 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1199/2423] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:07:55 2022] PUT /identity/v3/projects/0b47cdb6680246c7a93abd43b2b3641d/users/3d6f15ddbf1049eda9370e40bb31f857/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:55.138202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a28c0-0536-439e-9d02-7c3a1fcda8a0 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:55.138558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a28c0-0536-439e-9d02-7c3a1fcda8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:55.138903 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a28c0-0536-439e-9d02-7c3a1fcda8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:55.168270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9f1a28c0-0536-439e-9d02-7c3a1fcda8a0 None None] MFA Rules not processed for user `3d6f15ddbf1049eda9370e40bb31f857`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:55.169488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9f1a28c0-0536-439e-9d02-7c3a1fcda8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:55.194123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:55.236562 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1225/2424] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:55.241014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a271-6d36-4558-8de9-c8ce1c57bd87 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:55.304504 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a271-6d36-4558-8de9-c8ce1c57bd87 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:55.309209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daa37b-1e63-4534-8c12-a141eddcf791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:55.309826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daa37b-1e63-4534-8c12-a141eddcf791 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:55.310155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daa37b-1e63-4534-8c12-a141eddcf791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:55.310422 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daa37b-1e63-4534-8c12-a141eddcf791 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:55.311045 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32daa37b-1e63-4534-8c12-a141eddcf791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:55.313051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32daa37b-1e63-4534-8c12-a141eddcf791 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:55.323496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:55.343350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1200/2425] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:07:55 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:07:55.348335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a28c0-0536-439e-9d02-7c3a1fcda8a0 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:55.358950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a28c0-0536-439e-9d02-7c3a1fcda8a0 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:55.415451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e758be-7f64-41fc-95bc-f3590baca3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:55.416088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e758be-7f64-41fc-95bc-f3590baca3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:55.416371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e758be-7f64-41fc-95bc-f3590baca3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:55.416640 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e758be-7f64-41fc-95bc-f3590baca3b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:55.417266 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e758be-7f64-41fc-95bc-f3590baca3b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:55.419181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e758be-7f64-41fc-95bc-f3590baca3b9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:55.443566 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1226/2426] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:07:55 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:07:55.444966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daa37b-1e63-4534-8c12-a141eddcf791 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:55.455889 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daa37b-1e63-4534-8c12-a141eddcf791 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:55.460667 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477222c1-24ff-4d51-a75e-2f2904d3e9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:55.461354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477222c1-24ff-4d51-a75e-2f2904d3e9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:55.461700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477222c1-24ff-4d51-a75e-2f2904d3e9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:55.462088 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477222c1-24ff-4d51-a75e-2f2904d3e9c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:55.462796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477222c1-24ff-4d51-a75e-2f2904d3e9c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:55.464529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477222c1-24ff-4d51-a75e-2f2904d3e9c0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:55.475622 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1201/2427] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:07:55.479982 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e758be-7f64-41fc-95bc-f3590baca3b9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:55.491254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e758be-7f64-41fc-95bc-f3590baca3b9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:55.496026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d22456-ac9c-40c3-9c16-b749b41bfe44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:55.496673 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d22456-ac9c-40c3-9c16-b749b41bfe44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:55.496951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d22456-ac9c-40c3-9c16-b749b41bfe44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:55.497221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d22456-ac9c-40c3-9c16-b749b41bfe44 admin admin] PATH_INFO: `/v3/projects/6788d242dab24e338c3d258c49a7a775/users/57d3d605b5d1447a9250a111d4fde756/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:55.512268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d22456-ac9c-40c3-9c16-b749b41bfe44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6788d242dab24e338c3d258c49a7a775, user_id=57d3d605b5d1447a9250a111d4fde756, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:55.514928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d22456-ac9c-40c3-9c16-b749b41bfe44 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:55.529229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1227/2428] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:07:55 2022] PUT /identity/v3/projects/6788d242dab24e338c3d258c49a7a775/users/57d3d605b5d1447a9250a111d4fde756/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:55.534313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477222c1-24ff-4d51-a75e-2f2904d3e9c0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:55.544827 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477222c1-24ff-4d51-a75e-2f2904d3e9c0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:55.602638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f4620-a524-4f57-a4f0-ab0e1122c91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:55.603554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f4620-a524-4f57-a4f0-ab0e1122c91b admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:55.603971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f4620-a524-4f57-a4f0-ab0e1122c91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:55.604322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f4620-a524-4f57-a4f0-ab0e1122c91b admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:55.605115 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f4620-a524-4f57-a4f0-ab0e1122c91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:55.607139 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f4620-a524-4f57-a4f0-ab0e1122c91b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:55.617960 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1202/2429] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:07:55.622308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d22456-ac9c-40c3-9c16-b749b41bfe44 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:55.634294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d22456-ac9c-40c3-9c16-b749b41bfe44 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:55.639855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af374121-e051-4a75-b9ec-40c43ba11ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:55.640677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af374121-e051-4a75-b9ec-40c43ba11ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:55.641031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af374121-e051-4a75-b9ec-40c43ba11ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:55.642082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af374121-e051-4a75-b9ec-40c43ba11ea6 admin admin] PATH_INFO: `/v3/projects/6788d242dab24e338c3d258c49a7a775/users/57d3d605b5d1447a9250a111d4fde756/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:55.644749 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af374121-e051-4a75-b9ec-40c43ba11ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6788d242dab24e338c3d258c49a7a775, user_id=57d3d605b5d1447a9250a111d4fde756, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:55.648131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af374121-e051-4a75-b9ec-40c43ba11ea6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:55.661859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1228/2430] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:07:55 2022] PUT /identity/v3/projects/6788d242dab24e338c3d258c49a7a775/users/57d3d605b5d1447a9250a111d4fde756/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:55.672068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7397e06-7437-44bc-8157-3c4421ccdaf1 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:55.675733 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7397e06-7437-44bc-8157-3c4421ccdaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:55.676071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7397e06-7437-44bc-8157-3c4421ccdaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:55.707119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d7397e06-7437-44bc-8157-3c4421ccdaf1 None None] MFA Rules not processed for user `57d3d605b5d1447a9250a111d4fde756`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:55.708402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d7397e06-7437-44bc-8157-3c4421ccdaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:55.788444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1203/2431] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:55.803698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af374121-e051-4a75-b9ec-40c43ba11ea6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:55.805700 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-af374121-e051-4a75-b9ec-40c43ba11ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:55.872157 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af374121-e051-4a75-b9ec-40c43ba11ea6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:55.876238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8c452-b167-4f6a-b881-6a3fe988fc48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:55.877139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8c452-b167-4f6a-b881-6a3fe988fc48 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:55.878035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8c452-b167-4f6a-b881-6a3fe988fc48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:55.878311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8c452-b167-4f6a-b881-6a3fe988fc48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:55.887442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8c452-b167-4f6a-b881-6a3fe988fc48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:55.889010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8c452-b167-4f6a-b881-6a3fe988fc48 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:55.900056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1229/2432] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:07:56.338238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7397e06-7437-44bc-8157-3c4421ccdaf1 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:56.402897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7397e06-7437-44bc-8157-3c4421ccdaf1 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:56.407460 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:56.408127 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:56.408409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:56.408684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] PATH_INFO: `/v3/users/3d6f15ddbf1049eda9370e40bb31f857` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:56.409808 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d6f15ddbf1049eda9370e40bb31f857)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:56.411783 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:56.413395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:56.461634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d6f15ddbf1049eda9370e40bb31f857'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:56.469049 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d6f15ddbf1049eda9370e40bb31f857'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:56.481879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d6f15ddbf1049eda9370e40bb31f857'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:56.482596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d6f15ddbf1049eda9370e40bb31f857'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:56.483200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d6f15ddbf1049eda9370e40bb31f857'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:56.497190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1204/2433] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:07:56 2022] DELETE /identity/v3/users/3d6f15ddbf1049eda9370e40bb31f857 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:56.502484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d96821-0ae4-4c4d-ba51-ed200187eb1b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:56.502775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d96821-0ae4-4c4d-ba51-ed200187eb1b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:56.503052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d96821-0ae4-4c4d-ba51-ed200187eb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:56.522557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-44d96821-0ae4-4c4d-ba51-ed200187eb1b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:56.523775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-44d96821-0ae4-4c4d-ba51-ed200187eb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:56.537020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:56.590406 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1230/2434] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:56.601433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:56.602997 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:56.665081 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bed9d-5bdf-4447-a9d6-c7774a710861 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:56.668694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5beb1-6bae-40ab-8707-c4ed7674aa6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:56.669297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5beb1-6bae-40ab-8707-c4ed7674aa6d service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:56.669580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5beb1-6bae-40ab-8707-c4ed7674aa6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:56.669855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5beb1-6bae-40ab-8707-c4ed7674aa6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:56.679918 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5beb1-6bae-40ab-8707-c4ed7674aa6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:56.681387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5beb1-6bae-40ab-8707-c4ed7674aa6d service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:56.691165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1205/2435] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:07:56.993237 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d96821-0ae4-4c4d-ba51-ed200187eb1b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:56.995333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-44d96821-0ae4-4c4d-ba51-ed200187eb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:57.014789 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d96821-0ae4-4c4d-ba51-ed200187eb1b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:57.024568 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:57.025298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:57.025692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:57.026034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] PATH_INFO: `/v3/projects/0b47cdb6680246c7a93abd43b2b3641d` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:57.027643 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b47cdb6680246c7a93abd43b2b3641d)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:57.030346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:57.105510 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b47cdb6680246c7a93abd43b2b3641d', 'request_id': 'req-b4eca1ea-22ee-4951-8054-71246cec053d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.113940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b47cdb6680246c7a93abd43b2b3641d', 'request_id': 'req-b4eca1ea-22ee-4951-8054-71246cec053d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.119522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c4ed0-cbe4-49cc-8a9a-0566d6487915 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:57.119891 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c4ed0-cbe4-49cc-8a9a-0566d6487915 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:57.120292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c4ed0-cbe4-49cc-8a9a-0566d6487915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:57.121852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b47cdb6680246c7a93abd43b2b3641d', 'request_id': 'req-b4eca1ea-22ee-4951-8054-71246cec053d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.137346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] The token cache is being invalidate because project 0b47cdb6680246c7a93abd43b2b3641d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:07:57.137673 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.138708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.140261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1231/2436] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:07:56 2022] DELETE /identity/v3/projects/0b47cdb6680246c7a93abd43b2b3641d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:57.144649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:57.146154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:57.150013 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e97c4ed0-cbe4-49cc-8a9a-0566d6487915 None None] MFA Rules not processed for user `afd0a6d3b2194604a46767cfde12097b`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:57.152166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e97c4ed0-cbe4-49cc-8a9a-0566d6487915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:57.216297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1206/2437] 10.222.0.33 () {58 vars in 967 bytes} [Sun Sep 11 06:07:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:57.225420 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eca1ea-22ee-4951-8054-71246cec053d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:57.229664 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c4ed0-cbe4-49cc-8a9a-0566d6487915 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:57.230997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:57.231166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e97c4ed0-cbe4-49cc-8a9a-0566d6487915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:57.232041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:57.232487 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:57.232908 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] PATH_INFO: `/v3/users/57d3d605b5d1447a9250a111d4fde756` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:57.234190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57d3d605b5d1447a9250a111d4fde756)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:57.236509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:57.265825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:57.293775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:57.295895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57d3d605b5d1447a9250a111d4fde756'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.296683 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57d3d605b5d1447a9250a111d4fde756'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.306042 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57d3d605b5d1447a9250a111d4fde756'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.316554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57d3d605b5d1447a9250a111d4fde756'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.318014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c4ed0-cbe4-49cc-8a9a-0566d6487915 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:57.323368 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd983708-7057-43a3-a85b-4a55fc761121 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a726949e8cc4cfdb25aa78f2761cff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:57.323972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd983708-7057-43a3-a85b-4a55fc761121 service swift] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:57.324283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd983708-7057-43a3-a85b-4a55fc761121 service swift] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:57.324556 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd983708-7057-43a3-a85b-4a55fc761121 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:57.327840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bd983708-7057-43a3-a85b-4a55fc761121 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:57.332123 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57d3d605b5d1447a9250a111d4fde756'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.340605 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd983708-7057-43a3-a85b-4a55fc761121 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:57.342256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd983708-7057-43a3-a85b-4a55fc761121 service swift] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:57.346849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1232/2438] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:07:57 2022] DELETE /identity/v3/users/57d3d605b5d1447a9250a111d4fde756 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:57.351803 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1207/2439] 10.222.0.33 () {60 vars in 1384 bytes} [Sun Sep 11 06:07:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 06:07:57.669882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:57.671329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:57.792134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9978c-e077-4dac-90f1-51804c04e235 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:57.798471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44418491-0ea1-4621-8a25-e82edc15fff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:57.799249 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44418491-0ea1-4621-8a25-e82edc15fff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:57.799592 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44418491-0ea1-4621-8a25-e82edc15fff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:57.800042 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44418491-0ea1-4621-8a25-e82edc15fff5 admin admin] PATH_INFO: `/v3/projects/6788d242dab24e338c3d258c49a7a775` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:57.801640 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44418491-0ea1-4621-8a25-e82edc15fff5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6788d242dab24e338c3d258c49a7a775)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:57.803788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44418491-0ea1-4621-8a25-e82edc15fff5 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:57.854390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-44418491-0ea1-4621-8a25-e82edc15fff5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6788d242dab24e338c3d258c49a7a775', 'request_id': 'req-44418491-0ea1-4621-8a25-e82edc15fff5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.860823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-44418491-0ea1-4621-8a25-e82edc15fff5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6788d242dab24e338c3d258c49a7a775', 'request_id': 'req-44418491-0ea1-4621-8a25-e82edc15fff5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.867289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-44418491-0ea1-4621-8a25-e82edc15fff5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6788d242dab24e338c3d258c49a7a775', 'request_id': 'req-44418491-0ea1-4621-8a25-e82edc15fff5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.881249 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-44418491-0ea1-4621-8a25-e82edc15fff5 admin admin] The token cache is being invalidate because project 6788d242dab24e338c3d258c49a7a775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:07:57.881540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-44418491-0ea1-4621-8a25-e82edc15fff5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.882169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-44418491-0ea1-4621-8a25-e82edc15fff5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:57.883645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1233/2440] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:07:57 2022] DELETE /identity/v3/projects/6788d242dab24e338c3d258c49a7a775 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:57.888647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb66f0-e98c-4d21-8283-c0a545e795ba None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:57.888998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb66f0-e98c-4d21-8283-c0a545e795ba None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:57.889339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb66f0-e98c-4d21-8283-c0a545e795ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:57.909473 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-cecb66f0-e98c-4d21-8283-c0a545e795ba None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:57.910692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-cecb66f0-e98c-4d21-8283-c0a545e795ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:57.980980 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1208/2441] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:57.991104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ec0af-7088-4fa0-b38d-5cf9491b30d8 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:57.991392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ec0af-7088-4fa0-b38d-5cf9491b30d8 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:57.991660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ec0af-7088-4fa0-b38d-5cf9491b30d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.018827 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d67ec0af-7088-4fa0-b38d-5cf9491b30d8 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:58.022416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d67ec0af-7088-4fa0-b38d-5cf9491b30d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:58.035256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1234/2442] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:07:58.036536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb66f0-e98c-4d21-8283-c0a545e795ba None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:58.050042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb66f0-e98c-4d21-8283-c0a545e795ba None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:58.055281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43b58d-9b50-4035-b95d-2a8f4984ddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:58.055942 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43b58d-9b50-4035-b95d-2a8f4984ddf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.056387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43b58d-9b50-4035-b95d-2a8f4984ddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.056715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43b58d-9b50-4035-b95d-2a8f4984ddf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.057392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43b58d-9b50-4035-b95d-2a8f4984ddf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:07:58.057708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43b58d-9b50-4035-b95d-2a8f4984ddf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:58.060034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43b58d-9b50-4035-b95d-2a8f4984ddf5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:58.075841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1209/2443] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 06:07:58.080424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ec0af-7088-4fa0-b38d-5cf9491b30d8 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:58.091796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ec0af-7088-4fa0-b38d-5cf9491b30d8 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:58.098081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013e96f-0fbe-431a-bfcd-e4466daf2242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:58.098814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e96f-0fbe-431a-bfcd-e4466daf2242 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.099094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e96f-0fbe-431a-bfcd-e4466daf2242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.099365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e96f-0fbe-431a-bfcd-e4466daf2242 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.100377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013e96f-0fbe-431a-bfcd-e4466daf2242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:58.102640 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013e96f-0fbe-431a-bfcd-e4466daf2242 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:58.147209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1235/2444] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:07:58 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:07:58.151249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43b58d-9b50-4035-b95d-2a8f4984ddf5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:58.162169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43b58d-9b50-4035-b95d-2a8f4984ddf5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:58.235220 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75cd75-2982-4183-87bc-2ae02f349444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:58.235914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75cd75-2982-4183-87bc-2ae02f349444 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.236172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75cd75-2982-4183-87bc-2ae02f349444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.236444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75cd75-2982-4183-87bc-2ae02f349444 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.237074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75cd75-2982-4183-87bc-2ae02f349444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:58.239314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75cd75-2982-4183-87bc-2ae02f349444 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:58.280039 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1210/2445] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:07:58 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:07:58.284022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013e96f-0fbe-431a-bfcd-e4466daf2242 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:58.295099 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013e96f-0fbe-431a-bfcd-e4466daf2242 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:58.300290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9179b5-71c1-4751-aa70-edd67c678a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:58.300933 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9179b5-71c1-4751-aa70-edd67c678a6a admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.301288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9179b5-71c1-4751-aa70-edd67c678a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.301634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9179b5-71c1-4751-aa70-edd67c678a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.302382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9179b5-71c1-4751-aa70-edd67c678a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:58.304133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9179b5-71c1-4751-aa70-edd67c678a6a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:58.318756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1236/2446] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:07:58.320180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75cd75-2982-4183-87bc-2ae02f349444 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:58.332164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75cd75-2982-4183-87bc-2ae02f349444 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:58.337103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b148ab-e0f6-4e07-a749-1004b38634d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:58.337710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b148ab-e0f6-4e07-a749-1004b38634d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.337990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b148ab-e0f6-4e07-a749-1004b38634d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.338258 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b148ab-e0f6-4e07-a749-1004b38634d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.338896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b148ab-e0f6-4e07-a749-1004b38634d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:58.340623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b148ab-e0f6-4e07-a749-1004b38634d8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:58.352441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1211/2447] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:07:58.356686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9179b5-71c1-4751-aa70-edd67c678a6a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:58.367291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9179b5-71c1-4751-aa70-edd67c678a6a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:58.371956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649098cf-9539-461d-ade1-d12950d6253d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:58.372650 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649098cf-9539-461d-ade1-d12950d6253d admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.372935 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649098cf-9539-461d-ade1-d12950d6253d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.373210 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649098cf-9539-461d-ade1-d12950d6253d admin admin] PATH_INFO: `/v3/projects/b2185f11649640118a632689e1a2ef2e/users/418982c2d8db4044815608f635e63f37/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.386521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:58.390991 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649098cf-9539-461d-ade1-d12950d6253d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2185f11649640118a632689e1a2ef2e, user_id=418982c2d8db4044815608f635e63f37, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:58.395010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649098cf-9539-461d-ade1-d12950d6253d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:58.408864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1237/2448] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:07:58 2022] PUT /identity/v3/projects/b2185f11649640118a632689e1a2ef2e/users/418982c2d8db4044815608f635e63f37/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:58.424361 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffaea8-d494-437b-ad2d-ab018ea7ce20 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.424665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffaea8-d494-437b-ad2d-ab018ea7ce20 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.424958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffaea8-d494-437b-ad2d-ab018ea7ce20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.453354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c5ffaea8-d494-437b-ad2d-ab018ea7ce20 None None] MFA Rules not processed for user `418982c2d8db4044815608f635e63f37`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:58.455324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c5ffaea8-d494-437b-ad2d-ab018ea7ce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:58.465184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:58.523529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1212/2449] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:58.528077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649098cf-9539-461d-ade1-d12950d6253d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:58.590670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649098cf-9539-461d-ade1-d12950d6253d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:58.595424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d2103-c0a0-49ae-8229-8e68ece32a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:58.596068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2103-c0a0-49ae-8229-8e68ece32a9b admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.596354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2103-c0a0-49ae-8229-8e68ece32a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.596629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d2103-c0a0-49ae-8229-8e68ece32a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.597286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d2103-c0a0-49ae-8229-8e68ece32a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:58.599222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d2103-c0a0-49ae-8229-8e68ece32a9b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:58.631189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1238/2450] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:07:58 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:07:58.632668 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffaea8-d494-437b-ad2d-ab018ea7ce20 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:58.643872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffaea8-d494-437b-ad2d-ab018ea7ce20 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:58.702407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70433c46-9e90-49ce-8837-8947b46048b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:58.703009 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70433c46-9e90-49ce-8837-8947b46048b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.703293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70433c46-9e90-49ce-8837-8947b46048b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.703621 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70433c46-9e90-49ce-8837-8947b46048b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.704296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70433c46-9e90-49ce-8837-8947b46048b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:58.706237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70433c46-9e90-49ce-8837-8947b46048b9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:58.728016 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1213/2451] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:07:58 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:07:58.731887 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d2103-c0a0-49ae-8229-8e68ece32a9b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:58.743360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d2103-c0a0-49ae-8229-8e68ece32a9b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:58.748148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3a3e7-b3ed-4d79-970e-a31c96a4c9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:58.748766 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3a3e7-b3ed-4d79-970e-a31c96a4c9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.749051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3a3e7-b3ed-4d79-970e-a31c96a4c9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.749328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3a3e7-b3ed-4d79-970e-a31c96a4c9e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.750183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3a3e7-b3ed-4d79-970e-a31c96a4c9e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:58.752456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3a3e7-b3ed-4d79-970e-a31c96a4c9e1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:58.763911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1239/2452] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:07:58.768179 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70433c46-9e90-49ce-8837-8947b46048b9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:58.778684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70433c46-9e90-49ce-8837-8947b46048b9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:58.784075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d24bfc-41ff-4d25-b733-bb13a606c78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:58.784750 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d24bfc-41ff-4d25-b733-bb13a606c78f admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.785031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d24bfc-41ff-4d25-b733-bb13a606c78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.785305 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d24bfc-41ff-4d25-b733-bb13a606c78f admin admin] PATH_INFO: `/v3/projects/213eeb3ffdf44d8e8aeaf8480b10c320/users/ba3fcaabc7d7403f8e9a43c97c62971b/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.802444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d24bfc-41ff-4d25-b733-bb13a606c78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=213eeb3ffdf44d8e8aeaf8480b10c320, user_id=ba3fcaabc7d7403f8e9a43c97c62971b, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:58.805154 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d24bfc-41ff-4d25-b733-bb13a606c78f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:58.819053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1214/2453] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:07:58 2022] PUT /identity/v3/projects/213eeb3ffdf44d8e8aeaf8480b10c320/users/ba3fcaabc7d7403f8e9a43c97c62971b/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:58.823300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3a3e7-b3ed-4d79-970e-a31c96a4c9e1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:58.833482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3a3e7-b3ed-4d79-970e-a31c96a4c9e1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:58.890445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd708f67-3809-464c-812f-fb4d84265483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:58.890972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd708f67-3809-464c-812f-fb4d84265483 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.891302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd708f67-3809-464c-812f-fb4d84265483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.891725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd708f67-3809-464c-812f-fb4d84265483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.892507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd708f67-3809-464c-812f-fb4d84265483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:58.894119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd708f67-3809-464c-812f-fb4d84265483 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:58.905261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1240/2454] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:07:58.909935 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d24bfc-41ff-4d25-b733-bb13a606c78f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:58.920976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d24bfc-41ff-4d25-b733-bb13a606c78f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:58.926568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51dd77-3287-4e2a-aa9d-f1ea8c409138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:58.927238 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51dd77-3287-4e2a-aa9d-f1ea8c409138 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.927519 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51dd77-3287-4e2a-aa9d-f1ea8c409138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.927792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51dd77-3287-4e2a-aa9d-f1ea8c409138 admin admin] PATH_INFO: `/v3/projects/213eeb3ffdf44d8e8aeaf8480b10c320/users/ba3fcaabc7d7403f8e9a43c97c62971b/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.929756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51dd77-3287-4e2a-aa9d-f1ea8c409138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=213eeb3ffdf44d8e8aeaf8480b10c320, user_id=ba3fcaabc7d7403f8e9a43c97c62971b, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:58.932609 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51dd77-3287-4e2a-aa9d-f1ea8c409138 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:58.944446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1215/2455] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:07:58 2022] PUT /identity/v3/projects/213eeb3ffdf44d8e8aeaf8480b10c320/users/ba3fcaabc7d7403f8e9a43c97c62971b/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:07:58.956144 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4dea5-82d5-4555-95c1-f227fbbd0c3e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:58.956434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4dea5-82d5-4555-95c1-f227fbbd0c3e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:58.956709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4dea5-82d5-4555-95c1-f227fbbd0c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:58.985079 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-19b4dea5-82d5-4555-95c1-f227fbbd0c3e None None] MFA Rules not processed for user `ba3fcaabc7d7403f8e9a43c97c62971b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:58.986298 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-19b4dea5-82d5-4555-95c1-f227fbbd0c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:59.034002 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:59.057764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1241/2456] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:07:59.069692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51dd77-3287-4e2a-aa9d-f1ea8c409138 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:59.071462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ca51dd77-3287-4e2a-aa9d-f1ea8c409138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:59.131116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51dd77-3287-4e2a-aa9d-f1ea8c409138 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:59.134820 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1efd8-904f-498c-92de-65a17a886e1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:59.135435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1efd8-904f-498c-92de-65a17a886e1e service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:59.135715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1efd8-904f-498c-92de-65a17a886e1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:59.136016 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1efd8-904f-498c-92de-65a17a886e1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:59.146520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1efd8-904f-498c-92de-65a17a886e1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:59.147890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1efd8-904f-498c-92de-65a17a886e1e service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:59.158051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1216/2457] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:59 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:07:59.193588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4dea5-82d5-4555-95c1-f227fbbd0c3e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:59.258486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4dea5-82d5-4555-95c1-f227fbbd0c3e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:59.263005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:59.263624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:59.263905 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:59.264207 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] PATH_INFO: `/v3/users/418982c2d8db4044815608f635e63f37` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:59.265719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] RBAC: Authorizing `identity:delete_user(user_id=418982c2d8db4044815608f635e63f37)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:59.268016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:59.315894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '418982c2d8db4044815608f635e63f37'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:59.316582 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '418982c2d8db4044815608f635e63f37'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:59.323865 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '418982c2d8db4044815608f635e63f37'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:59.330710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '418982c2d8db4044815608f635e63f37'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:59.342658 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '418982c2d8db4044815608f635e63f37'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:59.348647 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1242/2458] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:07:59 2022] DELETE /identity/v3/users/418982c2d8db4044815608f635e63f37 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:59.353139 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcacbb-3209-4b36-bb03-3b145bb49a09 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:59.353688 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcacbb-3209-4b36-bb03-3b145bb49a09 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:59.353688 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcacbb-3209-4b36-bb03-3b145bb49a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:59.367298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:59.373533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-c5dcacbb-3209-4b36-bb03-3b145bb49a09 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:07:59.374814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c5dcacbb-3209-4b36-bb03-3b145bb49a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:59.437406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1217/2459] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:07:59.444605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:59.446162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:59.504287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66635f-ad84-4ccd-8703-c444068b367d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:59.508181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738e4ba-78b4-4a83-a8e0-2137868360a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:59.508782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738e4ba-78b4-4a83-a8e0-2137868360a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:59.509058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738e4ba-78b4-4a83-a8e0-2137868360a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:59.509328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738e4ba-78b4-4a83-a8e0-2137868360a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:59.518437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738e4ba-78b4-4a83-a8e0-2137868360a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:59.519844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738e4ba-78b4-4a83-a8e0-2137868360a7 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:59.529657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1243/2460] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:07:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:07:59.801351 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcacbb-3209-4b36-bb03-3b145bb49a09 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:59.802841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c5dcacbb-3209-4b36-bb03-3b145bb49a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:07:59.817695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcacbb-3209-4b36-bb03-3b145bb49a09 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:59.827664 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:59.828390 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:59.829309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:59.829591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] PATH_INFO: `/v3/projects/b2185f11649640118a632689e1a2ef2e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:59.831292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2185f11649640118a632689e1a2ef2e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:59.833282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:07:59.883242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2185f11649640118a632689e1a2ef2e', 'request_id': 'req-e674c771-f44c-4335-8ae1-67c44b081ee4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:59.890416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2185f11649640118a632689e1a2ef2e', 'request_id': 'req-e674c771-f44c-4335-8ae1-67c44b081ee4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:59.898183 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2185f11649640118a632689e1a2ef2e', 'request_id': 'req-e674c771-f44c-4335-8ae1-67c44b081ee4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:59.908939 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] The token cache is being invalidate because project b2185f11649640118a632689e1a2ef2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:07:59.909222 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:59.909872 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:07:59.911350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1218/2461] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:07:59 2022] DELETE /identity/v3/projects/b2185f11649640118a632689e1a2ef2e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:07:59.915356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738e4ba-78b4-4a83-a8e0-2137868360a7 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:07:59.925011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738e4ba-78b4-4a83-a8e0-2137868360a7 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:07:59.929045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:07:59.984336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20c406-132c-4e3e-9659-9a92375911fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:07:59.985476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20c406-132c-4e3e-9659-9a92375911fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:07:59.985754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20c406-132c-4e3e-9659-9a92375911fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:07:59.986107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20c406-132c-4e3e-9659-9a92375911fe admin admin] PATH_INFO: `/v3/users/ba3fcaabc7d7403f8e9a43c97c62971b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:07:59.987217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20c406-132c-4e3e-9659-9a92375911fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba3fcaabc7d7403f8e9a43c97c62971b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:07:59.989173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20c406-132c-4e3e-9659-9a92375911fe admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:00.037034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-cc20c406-132c-4e3e-9659-9a92375911fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba3fcaabc7d7403f8e9a43c97c62971b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:00.037717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-cc20c406-132c-4e3e-9659-9a92375911fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba3fcaabc7d7403f8e9a43c97c62971b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:00.045121 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-cc20c406-132c-4e3e-9659-9a92375911fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba3fcaabc7d7403f8e9a43c97c62971b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:00.053686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-cc20c406-132c-4e3e-9659-9a92375911fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba3fcaabc7d7403f8e9a43c97c62971b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:00.067260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-cc20c406-132c-4e3e-9659-9a92375911fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba3fcaabc7d7403f8e9a43c97c62971b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:00.073354 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1244/2462] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:07:59 2022] DELETE /identity/v3/users/ba3fcaabc7d7403f8e9a43c97c62971b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:00.357257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:00.358778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:00.374506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c771-f44c-4335-8ae1-67c44b081ee4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:00.448016 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5890d51-4090-4247-b47c-e94a39db30f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:00.448669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5890d51-4090-4247-b47c-e94a39db30f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:00.448985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5890d51-4090-4247-b47c-e94a39db30f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:00.449267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5890d51-4090-4247-b47c-e94a39db30f8 admin admin] PATH_INFO: `/v3/projects/213eeb3ffdf44d8e8aeaf8480b10c320` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:00.450406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5890d51-4090-4247-b47c-e94a39db30f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=213eeb3ffdf44d8e8aeaf8480b10c320)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:00.452426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5890d51-4090-4247-b47c-e94a39db30f8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:00.490204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:00.505022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c5890d51-4090-4247-b47c-e94a39db30f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213eeb3ffdf44d8e8aeaf8480b10c320', 'request_id': 'req-c5890d51-4090-4247-b47c-e94a39db30f8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:00.512827 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c5890d51-4090-4247-b47c-e94a39db30f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213eeb3ffdf44d8e8aeaf8480b10c320', 'request_id': 'req-c5890d51-4090-4247-b47c-e94a39db30f8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:00.519491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c5890d51-4090-4247-b47c-e94a39db30f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213eeb3ffdf44d8e8aeaf8480b10c320', 'request_id': 'req-c5890d51-4090-4247-b47c-e94a39db30f8'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:00.530529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c5890d51-4090-4247-b47c-e94a39db30f8 admin admin] The token cache is being invalidate because project 213eeb3ffdf44d8e8aeaf8480b10c320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:08:00.530818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c5890d51-4090-4247-b47c-e94a39db30f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:00.531464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c5890d51-4090-4247-b47c-e94a39db30f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:00.533628 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1219/2463] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:08:00 2022] DELETE /identity/v3/projects/213eeb3ffdf44d8e8aeaf8480b10c320 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:00.538587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a3e9-42f1-4275-9991-d449e348096e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:00.538872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a3e9-42f1-4275-9991-d449e348096e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:00.539145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a3e9-42f1-4275-9991-d449e348096e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:00.559258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bfd2a3e9-42f1-4275-9991-d449e348096e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:00.560585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bfd2a3e9-42f1-4275-9991-d449e348096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:00.626735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1245/2464] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:08:00.634589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e124ed7-ac0f-4bf7-b87a-2b601e34ccef None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:00.634879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e124ed7-ac0f-4bf7-b87a-2b601e34ccef None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:00.635154 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e124ed7-ac0f-4bf7-b87a-2b601e34ccef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:00.657236 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-2e124ed7-ac0f-4bf7-b87a-2b601e34ccef None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:00.658822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2e124ed7-ac0f-4bf7-b87a-2b601e34ccef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:00.666842 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1220/2465] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:08:00.671185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2a3e9-42f1-4275-9991-d449e348096e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:00.681596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2a3e9-42f1-4275-9991-d449e348096e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:00.686247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738a093-5083-4428-91dd-53ed2c091316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:00.686855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738a093-5083-4428-91dd-53ed2c091316 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:00.687136 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738a093-5083-4428-91dd-53ed2c091316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:00.687410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738a093-5083-4428-91dd-53ed2c091316 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:00.688366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738a093-5083-4428-91dd-53ed2c091316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:08:00.688770 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738a093-5083-4428-91dd-53ed2c091316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:00.690413 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738a093-5083-4428-91dd-53ed2c091316 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:00.707069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1246/2466] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:08:00.708628 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e124ed7-ac0f-4bf7-b87a-2b601e34ccef None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:00.718551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e124ed7-ac0f-4bf7-b87a-2b601e34ccef None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:00.724582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792ec8a-c14f-4ed7-8d4a-46771ae7c62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:00.725202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792ec8a-c14f-4ed7-8d4a-46771ae7c62c admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:00.725484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792ec8a-c14f-4ed7-8d4a-46771ae7c62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:00.725756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792ec8a-c14f-4ed7-8d4a-46771ae7c62c admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:00.726404 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792ec8a-c14f-4ed7-8d4a-46771ae7c62c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:00.728397 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792ec8a-c14f-4ed7-8d4a-46771ae7c62c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:00.758371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1221/2467] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:08:00 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:08:00.762544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738a093-5083-4428-91dd-53ed2c091316 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:00.773714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738a093-5083-4428-91dd-53ed2c091316 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:00.784179 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:00.845581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b1a83-35ca-4d5f-9f09-b4a331c9cf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:00.846198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b1a83-35ca-4d5f-9f09-b4a331c9cf62 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:00.846562 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b1a83-35ca-4d5f-9f09-b4a331c9cf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:00.846800 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b1a83-35ca-4d5f-9f09-b4a331c9cf62 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:00.847446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5b1a83-35ca-4d5f-9f09-b4a331c9cf62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:00.849456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5b1a83-35ca-4d5f-9f09-b4a331c9cf62 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:00.870991 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1247/2468] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:08:00 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:08:00.875097 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792ec8a-c14f-4ed7-8d4a-46771ae7c62c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:00.886531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792ec8a-c14f-4ed7-8d4a-46771ae7c62c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:00.892076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173b8bb-b995-4668-9c01-1cedfaaa26e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:00.892642 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173b8bb-b995-4668-9c01-1cedfaaa26e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:00.892923 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173b8bb-b995-4668-9c01-1cedfaaa26e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:00.893195 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173b8bb-b995-4668-9c01-1cedfaaa26e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:00.893843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173b8bb-b995-4668-9c01-1cedfaaa26e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:00.895493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173b8bb-b995-4668-9c01-1cedfaaa26e7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:00.906502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1222/2469] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:08:00.911664 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b1a83-35ca-4d5f-9f09-b4a331c9cf62 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:00.922000 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b1a83-35ca-4d5f-9f09-b4a331c9cf62 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:00.927577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480bc58-f62c-4d9d-8a44-3da14f893353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:00.928221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480bc58-f62c-4d9d-8a44-3da14f893353 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:00.928532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480bc58-f62c-4d9d-8a44-3da14f893353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:00.928804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480bc58-f62c-4d9d-8a44-3da14f893353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:00.929447 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480bc58-f62c-4d9d-8a44-3da14f893353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:00.931047 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480bc58-f62c-4d9d-8a44-3da14f893353 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:00.942805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1248/2470] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:08:00.947829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173b8bb-b995-4668-9c01-1cedfaaa26e7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:00.958759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173b8bb-b995-4668-9c01-1cedfaaa26e7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:00.963375 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3183307-8e90-44d0-99b1-9dc6b3e4f911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:00.964041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3183307-8e90-44d0-99b1-9dc6b3e4f911 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:00.964324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3183307-8e90-44d0-99b1-9dc6b3e4f911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:00.964592 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3183307-8e90-44d0-99b1-9dc6b3e4f911 admin admin] PATH_INFO: `/v3/projects/1e9ff028f7a74cd3b2066021ea450d98/users/35f46a673a40493a96eb20d99d1d425a/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:00.980647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3183307-8e90-44d0-99b1-9dc6b3e4f911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9ff028f7a74cd3b2066021ea450d98, user_id=35f46a673a40493a96eb20d99d1d425a, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:00.983295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3183307-8e90-44d0-99b1-9dc6b3e4f911 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:00.996672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1223/2471] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:08:00 2022] PUT /identity/v3/projects/1e9ff028f7a74cd3b2066021ea450d98/users/35f46a673a40493a96eb20d99d1d425a/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:01.009421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c834c9-6ea8-46fb-b050-754c5fc63fc5 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:01.009646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c834c9-6ea8-46fb-b050-754c5fc63fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:01.009922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c834c9-6ea8-46fb-b050-754c5fc63fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:01.041300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-02c834c9-6ea8-46fb-b050-754c5fc63fc5 None None] MFA Rules not processed for user `35f46a673a40493a96eb20d99d1d425a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:01.042511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-02c834c9-6ea8-46fb-b050-754c5fc63fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:01.113671 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1249/2472] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:08:01.118251 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3183307-8e90-44d0-99b1-9dc6b3e4f911 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:01.182681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3183307-8e90-44d0-99b1-9dc6b3e4f911 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:01.189092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7eebf6-9428-40fd-91c5-e81a529981bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:01.189827 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7eebf6-9428-40fd-91c5-e81a529981bc admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:01.190115 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7eebf6-9428-40fd-91c5-e81a529981bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:01.190391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7eebf6-9428-40fd-91c5-e81a529981bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:01.191028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7eebf6-9428-40fd-91c5-e81a529981bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:01.192993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7eebf6-9428-40fd-91c5-e81a529981bc admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:01.224555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1224/2473] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:08:01 2022] POST /identity/v3/projects => generated 389 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:08:01.228495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c834c9-6ea8-46fb-b050-754c5fc63fc5 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:01.240765 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c834c9-6ea8-46fb-b050-754c5fc63fc5 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:01.300438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0f5f2-df60-47d8-98ef-0ce7ff512a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:01.301116 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0f5f2-df60-47d8-98ef-0ce7ff512a70 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:01.301471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0f5f2-df60-47d8-98ef-0ce7ff512a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:01.301805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0f5f2-df60-47d8-98ef-0ce7ff512a70 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:01.302448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0f5f2-df60-47d8-98ef-0ce7ff512a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:01.304462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0f5f2-df60-47d8-98ef-0ce7ff512a70 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:01.327094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1250/2474] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:08:01 2022] POST /identity/v3/users => generated 301 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:08:01.333332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7eebf6-9428-40fd-91c5-e81a529981bc admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:01.347955 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7eebf6-9428-40fd-91c5-e81a529981bc admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:01.352982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700db14-9e55-4b95-855e-ae157780a4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:01.353586 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700db14-9e55-4b95-855e-ae157780a4fd admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:01.353895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700db14-9e55-4b95-855e-ae157780a4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:01.354167 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700db14-9e55-4b95-855e-ae157780a4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:01.354829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700db14-9e55-4b95-855e-ae157780a4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:01.356490 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700db14-9e55-4b95-855e-ae157780a4fd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:01.368072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1225/2475] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:08:01.372433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0f5f2-df60-47d8-98ef-0ce7ff512a70 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:01.383383 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0f5f2-df60-47d8-98ef-0ce7ff512a70 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:01.388545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e1328-53bb-4d83-8310-785b73ba8138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:01.389326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e1328-53bb-4d83-8310-785b73ba8138 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:01.389610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e1328-53bb-4d83-8310-785b73ba8138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:01.389878 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e1328-53bb-4d83-8310-785b73ba8138 admin admin] PATH_INFO: `/v3/projects/827a54d892904b0b9deca56a0a0b9742/users/793b923f92044eb5961747d4c021af2a/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:01.401108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:01.410739 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e1328-53bb-4d83-8310-785b73ba8138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=827a54d892904b0b9deca56a0a0b9742, user_id=793b923f92044eb5961747d4c021af2a, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:01.413422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e1328-53bb-4d83-8310-785b73ba8138 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:01.429306 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1251/2476] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:08:01 2022] PUT /identity/v3/projects/827a54d892904b0b9deca56a0a0b9742/users/793b923f92044eb5961747d4c021af2a/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:08:01.433632 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700db14-9e55-4b95-855e-ae157780a4fd admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:01.444910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700db14-9e55-4b95-855e-ae157780a4fd admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:01.475037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:01.506408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfec23-4b55-4f6d-b286-1324824bbe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:01.507023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfec23-4b55-4f6d-b286-1324824bbe84 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:01.507351 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfec23-4b55-4f6d-b286-1324824bbe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:01.507561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfec23-4b55-4f6d-b286-1324824bbe84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:01.508234 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfec23-4b55-4f6d-b286-1324824bbe84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:01.509841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfec23-4b55-4f6d-b286-1324824bbe84 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:01.520061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1226/2477] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:08:01.524630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e1328-53bb-4d83-8310-785b73ba8138 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:01.535057 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e1328-53bb-4d83-8310-785b73ba8138 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:01.539764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d628c8-f307-4fc6-83cc-f2b10a0b2d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:01.540452 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d628c8-f307-4fc6-83cc-f2b10a0b2d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:01.540731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d628c8-f307-4fc6-83cc-f2b10a0b2d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:01.541000 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d628c8-f307-4fc6-83cc-f2b10a0b2d05 admin admin] PATH_INFO: `/v3/projects/827a54d892904b0b9deca56a0a0b9742/users/793b923f92044eb5961747d4c021af2a/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:01.542909 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d628c8-f307-4fc6-83cc-f2b10a0b2d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=827a54d892904b0b9deca56a0a0b9742, user_id=793b923f92044eb5961747d4c021af2a, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:01.545578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d628c8-f307-4fc6-83cc-f2b10a0b2d05 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:01.558643 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1252/2478] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:08:01 2022] PUT /identity/v3/projects/827a54d892904b0b9deca56a0a0b9742/users/793b923f92044eb5961747d4c021af2a/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:01.570449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70900a03-e0c5-4d33-80fb-ccf6091aa049 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:01.570748 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70900a03-e0c5-4d33-80fb-ccf6091aa049 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:01.571023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70900a03-e0c5-4d33-80fb-ccf6091aa049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:01.599018 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-70900a03-e0c5-4d33-80fb-ccf6091aa049 None None] MFA Rules not processed for user `793b923f92044eb5961747d4c021af2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:01.600267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-70900a03-e0c5-4d33-80fb-ccf6091aa049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:01.671115 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1227/2479] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:01 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:08:01.682480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d628c8-f307-4fc6-83cc-f2b10a0b2d05 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:01.684250 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-09d628c8-f307-4fc6-83cc-f2b10a0b2d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:01.741361 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d628c8-f307-4fc6-83cc-f2b10a0b2d05 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:01.744994 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fab32-7900-466d-a58c-d933e699abeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:01.745630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fab32-7900-466d-a58c-d933e699abeb service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:01.745909 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fab32-7900-466d-a58c-d933e699abeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:01.746180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fab32-7900-466d-a58c-d933e699abeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:01.755294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fab32-7900-466d-a58c-d933e699abeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:01.756754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fab32-7900-466d-a58c-d933e699abeb service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:01.767279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1253/2480] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:08:01 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:08:01.908678 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70900a03-e0c5-4d33-80fb-ccf6091aa049 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:01.971213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70900a03-e0c5-4d33-80fb-ccf6091aa049 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:01.975692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:01.976380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:01.976658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:01.976928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] PATH_INFO: `/v3/users/35f46a673a40493a96eb20d99d1d425a` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:01.978041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f46a673a40493a96eb20d99d1d425a)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:01.979972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:02.029125 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f46a673a40493a96eb20d99d1d425a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.035448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:02.037806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f46a673a40493a96eb20d99d1d425a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.051365 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f46a673a40493a96eb20d99d1d425a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.052099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f46a673a40493a96eb20d99d1d425a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.052717 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f46a673a40493a96eb20d99d1d425a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.065557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1228/2481] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:08:01 2022] DELETE /identity/v3/users/35f46a673a40493a96eb20d99d1d425a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:02.070294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12922c7-ee20-407b-a5f1-17b9f5198243 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:02.070751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12922c7-ee20-407b-a5f1-17b9f5198243 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:02.070984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12922c7-ee20-407b-a5f1-17b9f5198243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:02.089585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c12922c7-ee20-407b-a5f1-17b9f5198243 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:02.090866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c12922c7-ee20-407b-a5f1-17b9f5198243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:02.157164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1254/2482] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:08:02.167485 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:02.169211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:02.232614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cdeb8-e988-4e3d-a9c3-f27a51e830bd admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:02.236962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb3042-422e-4117-b16d-3940f582cbaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:02.237574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb3042-422e-4117-b16d-3940f582cbaa service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:02.237853 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb3042-422e-4117-b16d-3940f582cbaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:02.238313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb3042-422e-4117-b16d-3940f582cbaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:02.247458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb3042-422e-4117-b16d-3940f582cbaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:02.248921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb3042-422e-4117-b16d-3940f582cbaa service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:02.258388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1229/2483] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:08:02.587358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12922c7-ee20-407b-a5f1-17b9f5198243 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:02.588900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c12922c7-ee20-407b-a5f1-17b9f5198243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:02.601610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12922c7-ee20-407b-a5f1-17b9f5198243 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:02.607334 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:02.607975 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:02.608281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:02.608552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] PATH_INFO: `/v3/projects/1e9ff028f7a74cd3b2066021ea450d98` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:02.610118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9ff028f7a74cd3b2066021ea450d98)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:02.611971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:02.661942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9ff028f7a74cd3b2066021ea450d98', 'request_id': 'req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.667915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9ff028f7a74cd3b2066021ea450d98', 'request_id': 'req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.674784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9ff028f7a74cd3b2066021ea450d98', 'request_id': 'req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.684154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:02.689523 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] The token cache is being invalidate because project 1e9ff028f7a74cd3b2066021ea450d98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:08:02.689995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.691345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.695829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1255/2484] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:08:02 2022] DELETE /identity/v3/projects/1e9ff028f7a74cd3b2066021ea450d98 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:02.697014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb3042-422e-4117-b16d-3940f582cbaa service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:02.707518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb3042-422e-4117-b16d-3940f582cbaa service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:02.765242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0fe4c-81d7-45aa-bd76-dd7440fa3ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:02.765786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0fe4c-81d7-45aa-bd76-dd7440fa3ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:02.766204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0fe4c-81d7-45aa-bd76-dd7440fa3ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:02.766474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0fe4c-81d7-45aa-bd76-dd7440fa3ad7 admin admin] PATH_INFO: `/v3/users/793b923f92044eb5961747d4c021af2a` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:02.767613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0fe4c-81d7-45aa-bd76-dd7440fa3ad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=793b923f92044eb5961747d4c021af2a)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:02.769585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0fe4c-81d7-45aa-bd76-dd7440fa3ad7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:02.828761 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-afe0fe4c-81d7-45aa-bd76-dd7440fa3ad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793b923f92044eb5961747d4c021af2a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.836904 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-afe0fe4c-81d7-45aa-bd76-dd7440fa3ad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793b923f92044eb5961747d4c021af2a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.850098 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-afe0fe4c-81d7-45aa-bd76-dd7440fa3ad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793b923f92044eb5961747d4c021af2a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.850868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-afe0fe4c-81d7-45aa-bd76-dd7440fa3ad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793b923f92044eb5961747d4c021af2a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.851591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-afe0fe4c-81d7-45aa-bd76-dd7440fa3ad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793b923f92044eb5961747d4c021af2a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:02.854643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:02.867161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1230/2485] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:08:02 2022] DELETE /identity/v3/users/793b923f92044eb5961747d4c021af2a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:03.162364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:03.164560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:03.180282 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc82884-9bf8-41d5-9304-8f9d53cd3afb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:03.241473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:03.242120 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:03.242399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:03.242670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382 admin admin] PATH_INFO: `/v3/projects/827a54d892904b0b9deca56a0a0b9742` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:03.243776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=827a54d892904b0b9deca56a0a0b9742)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:03.245721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:03.293448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '827a54d892904b0b9deca56a0a0b9742', 'request_id': 'req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:03.299708 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '827a54d892904b0b9deca56a0a0b9742', 'request_id': 'req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:03.306038 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '827a54d892904b0b9deca56a0a0b9742', 'request_id': 'req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:03.319234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382 admin admin] The token cache is being invalidate because project 827a54d892904b0b9deca56a0a0b9742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:08:03.319521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:03.320193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f638e24c-68a8-4eb2-a5ea-68d0ac8e8382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:03.321852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1256/2486] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:08:03 2022] DELETE /identity/v3/projects/827a54d892904b0b9deca56a0a0b9742 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:03.327109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e025f5-685f-48bb-b6e3-56394757b7e3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:03.327542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e025f5-685f-48bb-b6e3-56394757b7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:03.327873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e025f5-685f-48bb-b6e3-56394757b7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:03.350143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-72e025f5-685f-48bb-b6e3-56394757b7e3 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:03.351451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-72e025f5-685f-48bb-b6e3-56394757b7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:03.413943 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1231/2487] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:08:03.421540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17e3df-4963-4fbc-8925-105ba1d3bea7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:03.421826 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17e3df-4963-4fbc-8925-105ba1d3bea7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:03.422098 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17e3df-4963-4fbc-8925-105ba1d3bea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:03.447956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5c17e3df-4963-4fbc-8925-105ba1d3bea7 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:03.449195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5c17e3df-4963-4fbc-8925-105ba1d3bea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:03.457495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1257/2488] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:08:03.461819 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e025f5-685f-48bb-b6e3-56394757b7e3 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:03.472524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e025f5-685f-48bb-b6e3-56394757b7e3 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:03.477356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e18b1-e266-4f6a-bb07-500e2a3bd81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:03.477962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e18b1-e266-4f6a-bb07-500e2a3bd81d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:03.478314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e18b1-e266-4f6a-bb07-500e2a3bd81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:03.478589 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e18b1-e266-4f6a-bb07-500e2a3bd81d admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:03.479244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e18b1-e266-4f6a-bb07-500e2a3bd81d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:08:03.479571 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e18b1-e266-4f6a-bb07-500e2a3bd81d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:03.481218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e18b1-e266-4f6a-bb07-500e2a3bd81d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:03.494725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1232/2489] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:08:03.499176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17e3df-4963-4fbc-8925-105ba1d3bea7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:03.508819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17e3df-4963-4fbc-8925-105ba1d3bea7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:03.513639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1f5c8-20b7-4ff1-b1cb-f206eab30610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:03.514240 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f5c8-20b7-4ff1-b1cb-f206eab30610 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:03.514522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f5c8-20b7-4ff1-b1cb-f206eab30610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:03.514796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f5c8-20b7-4ff1-b1cb-f206eab30610 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:03.515436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1f5c8-20b7-4ff1-b1cb-f206eab30610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:03.517544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1f5c8-20b7-4ff1-b1cb-f206eab30610 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:03.545553 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1258/2490] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:08:03 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:08:03.550294 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e18b1-e266-4f6a-bb07-500e2a3bd81d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:03.568144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e18b1-e266-4f6a-bb07-500e2a3bd81d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:03.629529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84674831-d666-4234-87c2-993f414924f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:03.630134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84674831-d666-4234-87c2-993f414924f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:03.630416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84674831-d666-4234-87c2-993f414924f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:03.630684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84674831-d666-4234-87c2-993f414924f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:03.631318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84674831-d666-4234-87c2-993f414924f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:03.633280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84674831-d666-4234-87c2-993f414924f7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:03.655797 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1233/2491] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:08:03 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:08:03.660502 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1f5c8-20b7-4ff1-b1cb-f206eab30610 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:03.671796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1f5c8-20b7-4ff1-b1cb-f206eab30610 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:03.679550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c03a80-8550-41d8-8f50-c13f56b3e551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:03.680194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c03a80-8550-41d8-8f50-c13f56b3e551 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:03.680475 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c03a80-8550-41d8-8f50-c13f56b3e551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:03.680752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c03a80-8550-41d8-8f50-c13f56b3e551 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:03.681389 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c03a80-8550-41d8-8f50-c13f56b3e551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:03.683011 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c03a80-8550-41d8-8f50-c13f56b3e551 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:03.702761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1259/2492] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:08:03.704933 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84674831-d666-4234-87c2-993f414924f7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:03.722498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84674831-d666-4234-87c2-993f414924f7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:03.727885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93bfe1-cd0c-4511-bbd5-21e015378ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:03.728780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93bfe1-cd0c-4511-bbd5-21e015378ede admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:03.729202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93bfe1-cd0c-4511-bbd5-21e015378ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:03.729670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93bfe1-cd0c-4511-bbd5-21e015378ede admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:03.731232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93bfe1-cd0c-4511-bbd5-21e015378ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:03.733256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93bfe1-cd0c-4511-bbd5-21e015378ede admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:03.748372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1234/2493] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:08:03.751823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c03a80-8550-41d8-8f50-c13f56b3e551 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:03.762311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c03a80-8550-41d8-8f50-c13f56b3e551 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:03.767289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1cca9-e57c-47dd-b593-35abc0233816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:03.767945 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1cca9-e57c-47dd-b593-35abc0233816 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:03.768262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1cca9-e57c-47dd-b593-35abc0233816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:03.768539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1cca9-e57c-47dd-b593-35abc0233816 admin admin] PATH_INFO: `/v3/projects/24350fa202464ad5b531cd3ddc6bd335/users/b4bbd15dfcc043e39e616fd3a46f443e/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:03.783561 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1cca9-e57c-47dd-b593-35abc0233816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24350fa202464ad5b531cd3ddc6bd335, user_id=b4bbd15dfcc043e39e616fd3a46f443e, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:03.786268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1cca9-e57c-47dd-b593-35abc0233816 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:03.801760 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1260/2494] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:08:03 2022] PUT /identity/v3/projects/24350fa202464ad5b531cd3ddc6bd335/users/b4bbd15dfcc043e39e616fd3a46f443e/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:08:03.812014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57df458-b5f4-4002-a855-3653882b989d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:03.812318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57df458-b5f4-4002-a855-3653882b989d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:03.812590 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57df458-b5f4-4002-a855-3653882b989d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:03.844509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f57df458-b5f4-4002-a855-3653882b989d None None] MFA Rules not processed for user `b4bbd15dfcc043e39e616fd3a46f443e`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:03.845894 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f57df458-b5f4-4002-a855-3653882b989d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:03.893004 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:03.912588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1235/2495] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:08:03.917021 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1cca9-e57c-47dd-b593-35abc0233816 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:03.947778 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:03.981283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1cca9-e57c-47dd-b593-35abc0233816 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:03.986937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74769496-3b73-43ae-ad6c-1214f8572d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:03.987625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74769496-3b73-43ae-ad6c-1214f8572d72 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:03.988013 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74769496-3b73-43ae-ad6c-1214f8572d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:03.988311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74769496-3b73-43ae-ad6c-1214f8572d72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:03.988943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74769496-3b73-43ae-ad6c-1214f8572d72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:03.990863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74769496-3b73-43ae-ad6c-1214f8572d72 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:04.018453 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1261/2496] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:08:03 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:08:04.022976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57df458-b5f4-4002-a855-3653882b989d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:04.034072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57df458-b5f4-4002-a855-3653882b989d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:04.093685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55960cb-8d69-470b-aac4-aeb33667cd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:04.094327 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55960cb-8d69-470b-aac4-aeb33667cd87 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:04.094612 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55960cb-8d69-470b-aac4-aeb33667cd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:04.094885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55960cb-8d69-470b-aac4-aeb33667cd87 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:04.095516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55960cb-8d69-470b-aac4-aeb33667cd87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:04.097514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55960cb-8d69-470b-aac4-aeb33667cd87 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:04.122413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1236/2497] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:08:04 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:08:04.124509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74769496-3b73-43ae-ad6c-1214f8572d72 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:04.136270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74769496-3b73-43ae-ad6c-1214f8572d72 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:04.140923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3744ecb-f52f-46ef-9f45-da6720245a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:04.141697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3744ecb-f52f-46ef-9f45-da6720245a28 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:04.141982 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3744ecb-f52f-46ef-9f45-da6720245a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:04.142257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3744ecb-f52f-46ef-9f45-da6720245a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:04.142926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3744ecb-f52f-46ef-9f45-da6720245a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:04.144563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3744ecb-f52f-46ef-9f45-da6720245a28 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:04.156340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1262/2498] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:08:04.160976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55960cb-8d69-470b-aac4-aeb33667cd87 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:04.172757 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55960cb-8d69-470b-aac4-aeb33667cd87 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:04.180042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe142a3d-21a7-4471-a1d6-b8a61197739c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:04.180042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe142a3d-21a7-4471-a1d6-b8a61197739c admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:04.180042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe142a3d-21a7-4471-a1d6-b8a61197739c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:04.180377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe142a3d-21a7-4471-a1d6-b8a61197739c admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:04.182132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe142a3d-21a7-4471-a1d6-b8a61197739c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:04.183777 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe142a3d-21a7-4471-a1d6-b8a61197739c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:04.199014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1237/2499] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:08:04.204949 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3744ecb-f52f-46ef-9f45-da6720245a28 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:04.218980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3744ecb-f52f-46ef-9f45-da6720245a28 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:04.223955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418884d7-a846-4c5b-aeae-4d27c640fe61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:04.224637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418884d7-a846-4c5b-aeae-4d27c640fe61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:04.224913 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418884d7-a846-4c5b-aeae-4d27c640fe61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:04.225278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418884d7-a846-4c5b-aeae-4d27c640fe61 admin admin] PATH_INFO: `/v3/projects/674f5cf881e94c93829f8dc636730cd5/users/411f9d3c761c40369e6dd78a80bc5978/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:04.241195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418884d7-a846-4c5b-aeae-4d27c640fe61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=674f5cf881e94c93829f8dc636730cd5, user_id=411f9d3c761c40369e6dd78a80bc5978, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:04.244207 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418884d7-a846-4c5b-aeae-4d27c640fe61 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:04.259835 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1263/2500] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:08:04 2022] PUT /identity/v3/projects/674f5cf881e94c93829f8dc636730cd5/users/411f9d3c761c40369e6dd78a80bc5978/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:04.272159 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4356d-1183-4bc6-ab0d-94fce71e3d9b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:04.272444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4356d-1183-4bc6-ab0d-94fce71e3d9b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:04.272712 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4356d-1183-4bc6-ab0d-94fce71e3d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:04.301984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4df4356d-1183-4bc6-ab0d-94fce71e3d9b None None] MFA Rules not processed for user `411f9d3c761c40369e6dd78a80bc5978`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:04.303176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4df4356d-1183-4bc6-ab0d-94fce71e3d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:04.367901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1238/2501] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:08:04.372538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418884d7-a846-4c5b-aeae-4d27c640fe61 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:04.434336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418884d7-a846-4c5b-aeae-4d27c640fe61 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:04.439181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f8c71-ae04-47a1-ac82-855f61c4b6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:04.439783 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f8c71-ae04-47a1-ac82-855f61c4b6a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:04.440103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f8c71-ae04-47a1-ac82-855f61c4b6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:04.440377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f8c71-ae04-47a1-ac82-855f61c4b6a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:04.441014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f8c71-ae04-47a1-ac82-855f61c4b6a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:04.443171 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f8c71-ae04-47a1-ac82-855f61c4b6a8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:04.475289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1264/2502] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:08:04 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:08:04.479422 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4356d-1183-4bc6-ab0d-94fce71e3d9b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:04.491459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4356d-1183-4bc6-ab0d-94fce71e3d9b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:04.563551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:04.570145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c615b9-b9d6-4d93-9b47-67707d0217d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:04.570792 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c615b9-b9d6-4d93-9b47-67707d0217d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:04.571335 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c615b9-b9d6-4d93-9b47-67707d0217d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:04.571650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c615b9-b9d6-4d93-9b47-67707d0217d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:04.572633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c615b9-b9d6-4d93-9b47-67707d0217d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:04.574702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c615b9-b9d6-4d93-9b47-67707d0217d0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:04.598435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1239/2503] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:08:04 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:08:04.602743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f8c71-ae04-47a1-ac82-855f61c4b6a8 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:04.616316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f8c71-ae04-47a1-ac82-855f61c4b6a8 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:04.621113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34e0a8-5d84-4126-935b-59ff46828e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:04.621758 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34e0a8-5d84-4126-935b-59ff46828e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:04.622049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34e0a8-5d84-4126-935b-59ff46828e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:04.622314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34e0a8-5d84-4126-935b-59ff46828e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:04.622954 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34e0a8-5d84-4126-935b-59ff46828e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:04.624607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34e0a8-5d84-4126-935b-59ff46828e27 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:04.637307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1265/2504] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:08:04.641625 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c615b9-b9d6-4d93-9b47-67707d0217d0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:04.652737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c615b9-b9d6-4d93-9b47-67707d0217d0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:04.657800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af331ea-ea2c-45b0-9a78-d94cb285ca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:04.658465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af331ea-ea2c-45b0-9a78-d94cb285ca87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:04.658744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af331ea-ea2c-45b0-9a78-d94cb285ca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:04.659042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af331ea-ea2c-45b0-9a78-d94cb285ca87 admin admin] PATH_INFO: `/v3/projects/2f7dd41bd7534a4c960291d27844a469/users/c75f0c7602864a8f932424c339b36ec2/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:04.676812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af331ea-ea2c-45b0-9a78-d94cb285ca87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f7dd41bd7534a4c960291d27844a469, user_id=c75f0c7602864a8f932424c339b36ec2, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:04.679521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af331ea-ea2c-45b0-9a78-d94cb285ca87 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:04.695199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1240/2505] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:08:04 2022] PUT /identity/v3/projects/2f7dd41bd7534a4c960291d27844a469/users/c75f0c7602864a8f932424c339b36ec2/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:04.699355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34e0a8-5d84-4126-935b-59ff46828e27 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:04.710439 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34e0a8-5d84-4126-935b-59ff46828e27 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:04.762971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:04.775720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e94589-24d9-45b0-8faa-338f61575db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:04.776359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e94589-24d9-45b0-8faa-338f61575db5 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:04.776635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e94589-24d9-45b0-8faa-338f61575db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:04.776902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e94589-24d9-45b0-8faa-338f61575db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:04.777541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e94589-24d9-45b0-8faa-338f61575db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:04.779146 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e94589-24d9-45b0-8faa-338f61575db5 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:04.791498 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1266/2506] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:08:04.795804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af331ea-ea2c-45b0-9a78-d94cb285ca87 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:04.806524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af331ea-ea2c-45b0-9a78-d94cb285ca87 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:04.811569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41352258-5650-4bde-be61-516d7419de84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:04.812275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41352258-5650-4bde-be61-516d7419de84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:04.812557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41352258-5650-4bde-be61-516d7419de84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:04.812858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41352258-5650-4bde-be61-516d7419de84 admin admin] PATH_INFO: `/v3/projects/2f7dd41bd7534a4c960291d27844a469/users/c75f0c7602864a8f932424c339b36ec2/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:04.814945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41352258-5650-4bde-be61-516d7419de84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f7dd41bd7534a4c960291d27844a469, user_id=c75f0c7602864a8f932424c339b36ec2, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:04.817709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41352258-5650-4bde-be61-516d7419de84 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:04.830025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1241/2507] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:08:04 2022] PUT /identity/v3/projects/2f7dd41bd7534a4c960291d27844a469/users/c75f0c7602864a8f932424c339b36ec2/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:04.840087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6577d43-3adf-4eef-bfda-4984b064315f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:04.840394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6577d43-3adf-4eef-bfda-4984b064315f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:04.840752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6577d43-3adf-4eef-bfda-4984b064315f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:04.870596 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c6577d43-3adf-4eef-bfda-4984b064315f None None] MFA Rules not processed for user `c75f0c7602864a8f932424c339b36ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:04.871786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c6577d43-3adf-4eef-bfda-4984b064315f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:04.940997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1267/2508] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:08:04.957816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41352258-5650-4bde-be61-516d7419de84 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:04.959635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-41352258-5650-4bde-be61-516d7419de84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:05.018474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41352258-5650-4bde-be61-516d7419de84 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:05.022762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580a55e-6e6e-4ac2-8c19-d4a465a48504 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:05.023372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580a55e-6e6e-4ac2-8c19-d4a465a48504 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:05.023655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580a55e-6e6e-4ac2-8c19-d4a465a48504 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:05.023951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580a55e-6e6e-4ac2-8c19-d4a465a48504 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:05.034496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f580a55e-6e6e-4ac2-8c19-d4a465a48504 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:05.036089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f580a55e-6e6e-4ac2-8c19-d4a465a48504 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:05.046058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1242/2509] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:08:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:08:05.088935 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6577d43-3adf-4eef-bfda-4984b064315f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:05.090433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c6577d43-3adf-4eef-bfda-4984b064315f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:05.104391 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6577d43-3adf-4eef-bfda-4984b064315f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:05.108084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca72bc0-8a66-434f-b407-96b60d5db3b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:05.108691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca72bc0-8a66-434f-b407-96b60d5db3b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:05.108995 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca72bc0-8a66-434f-b407-96b60d5db3b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:05.109267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca72bc0-8a66-434f-b407-96b60d5db3b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:05.168336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca72bc0-8a66-434f-b407-96b60d5db3b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:05.169786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca72bc0-8a66-434f-b407-96b60d5db3b8 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:05.179754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1268/2510] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:08:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:08:08.092230 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580a55e-6e6e-4ac2-8c19-d4a465a48504 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:08.156920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580a55e-6e6e-4ac2-8c19-d4a465a48504 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:08.162036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:08.162670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:08.162946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:08.163216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] PATH_INFO: `/v3/users/b4bbd15dfcc043e39e616fd3a46f443e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:08.164538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4bbd15dfcc043e39e616fd3a46f443e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:08.166394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:08.224015 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4bbd15dfcc043e39e616fd3a46f443e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:08.229708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:08.232383 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4bbd15dfcc043e39e616fd3a46f443e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:08.246875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4bbd15dfcc043e39e616fd3a46f443e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:08.247617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4bbd15dfcc043e39e616fd3a46f443e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:08.248264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4bbd15dfcc043e39e616fd3a46f443e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:08.261382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1243/2511] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:08:08 2022] DELETE /identity/v3/users/b4bbd15dfcc043e39e616fd3a46f443e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:08.265860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e8103-1090-4ebf-a660-490103727f08 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:08.266320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e8103-1090-4ebf-a660-490103727f08 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:08.266431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e8103-1090-4ebf-a660-490103727f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:08.285603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bc5e8103-1090-4ebf-a660-490103727f08 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:08.287094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bc5e8103-1090-4ebf-a660-490103727f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:08.346199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1269/2512] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:08:08.356039 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:08.357680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:08.416694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f196ee-fb5f-40b6-a362-bd60ded66658 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:08.420489 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda32854-38ef-428b-95d0-f45cdd125398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:08.421091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda32854-38ef-428b-95d0-f45cdd125398 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:08.421369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda32854-38ef-428b-95d0-f45cdd125398 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:08.421638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda32854-38ef-428b-95d0-f45cdd125398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:08.431455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda32854-38ef-428b-95d0-f45cdd125398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:08.432944 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda32854-38ef-428b-95d0-f45cdd125398 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:08.443216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1244/2513] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:08:08.965134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e8103-1090-4ebf-a660-490103727f08 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:08.966607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bc5e8103-1090-4ebf-a660-490103727f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:08.989637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e8103-1090-4ebf-a660-490103727f08 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:08.995327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:08.996421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:08.999516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:08.999831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] PATH_INFO: `/v3/projects/24350fa202464ad5b531cd3ddc6bd335` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:09.001084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=24350fa202464ad5b531cd3ddc6bd335)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:09.003180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:09.010127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:09.053325 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24350fa202464ad5b531cd3ddc6bd335', 'request_id': 'req-8f4edd25-c137-4947-a681-73b3dc0a0e4d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.059077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24350fa202464ad5b531cd3ddc6bd335', 'request_id': 'req-8f4edd25-c137-4947-a681-73b3dc0a0e4d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.066226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24350fa202464ad5b531cd3ddc6bd335', 'request_id': 'req-8f4edd25-c137-4947-a681-73b3dc0a0e4d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.080494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] The token cache is being invalidate because project 24350fa202464ad5b531cd3ddc6bd335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:08:09.080779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.081411 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.082884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1270/2514] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:08:08 2022] DELETE /identity/v3/projects/24350fa202464ad5b531cd3ddc6bd335 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:09.087096 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda32854-38ef-428b-95d0-f45cdd125398 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:09.097142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda32854-38ef-428b-95d0-f45cdd125398 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:09.156883 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:09.157548 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:09.157825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:09.158132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] PATH_INFO: `/v3/users/411f9d3c761c40369e6dd78a80bc5978` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:09.159371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=411f9d3c761c40369e6dd78a80bc5978)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:09.161320 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:09.209211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '411f9d3c761c40369e6dd78a80bc5978'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.216160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '411f9d3c761c40369e6dd78a80bc5978'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.229332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '411f9d3c761c40369e6dd78a80bc5978'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.230048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '411f9d3c761c40369e6dd78a80bc5978'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.230653 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '411f9d3c761c40369e6dd78a80bc5978'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.233263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:09.246188 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1245/2515] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:08:09 2022] DELETE /identity/v3/users/411f9d3c761c40369e6dd78a80bc5978 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:09.526254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:09.527829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:09.544028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4edd25-c137-4947-a681-73b3dc0a0e4d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:09.604621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:09.605218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:09.605500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:09.605770 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] PATH_INFO: `/v3/projects/674f5cf881e94c93829f8dc636730cd5` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:09.606921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] RBAC: Authorizing `identity:delete_project(project_id=674f5cf881e94c93829f8dc636730cd5)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:09.609244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:09.660194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '674f5cf881e94c93829f8dc636730cd5', 'request_id': 'req-ae86a422-eb61-44e8-9e4b-272c1657e56e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.665943 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '674f5cf881e94c93829f8dc636730cd5', 'request_id': 'req-ae86a422-eb61-44e8-9e4b-272c1657e56e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.672336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '674f5cf881e94c93829f8dc636730cd5', 'request_id': 'req-ae86a422-eb61-44e8-9e4b-272c1657e56e'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.685968 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] The token cache is being invalidate because project 674f5cf881e94c93829f8dc636730cd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:08:09.686259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.686891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.688392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1271/2516] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:08:09 2022] DELETE /identity/v3/projects/674f5cf881e94c93829f8dc636730cd5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:09.692613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:09.694192 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:09.707810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f671279-e4fc-432d-90d7-de6819f21960 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:09.768849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84842db1-9055-498b-a5b6-bc71a9da537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:09.769484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84842db1-9055-498b-a5b6-bc71a9da537a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:09.769764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84842db1-9055-498b-a5b6-bc71a9da537a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:09.770042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84842db1-9055-498b-a5b6-bc71a9da537a admin admin] PATH_INFO: `/v3/users/c75f0c7602864a8f932424c339b36ec2` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:09.771184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84842db1-9055-498b-a5b6-bc71a9da537a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c75f0c7602864a8f932424c339b36ec2)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:09.773415 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84842db1-9055-498b-a5b6-bc71a9da537a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:09.835182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-84842db1-9055-498b-a5b6-bc71a9da537a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c75f0c7602864a8f932424c339b36ec2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.846951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-84842db1-9055-498b-a5b6-bc71a9da537a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c75f0c7602864a8f932424c339b36ec2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.859108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-84842db1-9055-498b-a5b6-bc71a9da537a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c75f0c7602864a8f932424c339b36ec2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.859770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-84842db1-9055-498b-a5b6-bc71a9da537a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c75f0c7602864a8f932424c339b36ec2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.860407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-84842db1-9055-498b-a5b6-bc71a9da537a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c75f0c7602864a8f932424c339b36ec2'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:09.872764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1246/2517] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:08:09 2022] DELETE /identity/v3/users/c75f0c7602864a8f932424c339b36ec2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:10.151001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:10.152540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:10.170233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86a422-eb61-44e8-9e4b-272c1657e56e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:10.194275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:10.234879 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42beab94-8df2-4a14-a489-991aeed5781d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:10.235574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42beab94-8df2-4a14-a489-991aeed5781d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:10.235987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42beab94-8df2-4a14-a489-991aeed5781d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:10.236471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42beab94-8df2-4a14-a489-991aeed5781d admin admin] PATH_INFO: `/v3/projects/2f7dd41bd7534a4c960291d27844a469` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:10.237624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42beab94-8df2-4a14-a489-991aeed5781d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f7dd41bd7534a4c960291d27844a469)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:10.239537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42beab94-8df2-4a14-a489-991aeed5781d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:10.289590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-42beab94-8df2-4a14-a489-991aeed5781d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f7dd41bd7534a4c960291d27844a469', 'request_id': 'req-42beab94-8df2-4a14-a489-991aeed5781d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:10.295515 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-42beab94-8df2-4a14-a489-991aeed5781d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f7dd41bd7534a4c960291d27844a469', 'request_id': 'req-42beab94-8df2-4a14-a489-991aeed5781d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:10.302181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-42beab94-8df2-4a14-a489-991aeed5781d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f7dd41bd7534a4c960291d27844a469', 'request_id': 'req-42beab94-8df2-4a14-a489-991aeed5781d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:10.316200 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-42beab94-8df2-4a14-a489-991aeed5781d admin admin] The token cache is being invalidate because project 2f7dd41bd7534a4c960291d27844a469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:08:10.316525 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-42beab94-8df2-4a14-a489-991aeed5781d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:10.317152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-42beab94-8df2-4a14-a489-991aeed5781d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:10.318657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1272/2518] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:08:10 2022] DELETE /identity/v3/projects/2f7dd41bd7534a4c960291d27844a469 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:10.324671 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068c1e0-31cc-41f3-ba7c-a67e10210454 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:10.325124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068c1e0-31cc-41f3-ba7c-a67e10210454 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:10.325240 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068c1e0-31cc-41f3-ba7c-a67e10210454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:10.347677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4068c1e0-31cc-41f3-ba7c-a67e10210454 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:10.348925 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4068c1e0-31cc-41f3-ba7c-a67e10210454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:10.409949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1247/2519] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:08:10.418695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad6ec4-0427-48a7-9a7b-1c611b99936d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:10.418984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad6ec4-0427-48a7-9a7b-1c611b99936d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:10.419259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad6ec4-0427-48a7-9a7b-1c611b99936d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:10.440261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f4ad6ec4-0427-48a7-9a7b-1c611b99936d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:10.441499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f4ad6ec4-0427-48a7-9a7b-1c611b99936d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:10.450102 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1273/2520] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:08:10.454089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068c1e0-31cc-41f3-ba7c-a67e10210454 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:10.464876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068c1e0-31cc-41f3-ba7c-a67e10210454 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:10.469830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac42c31-82e8-43dd-a856-01f4819ea710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:10.470967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac42c31-82e8-43dd-a856-01f4819ea710 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:10.471250 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac42c31-82e8-43dd-a856-01f4819ea710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:10.471522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac42c31-82e8-43dd-a856-01f4819ea710 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:10.472206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac42c31-82e8-43dd-a856-01f4819ea710 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:08:10.472513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac42c31-82e8-43dd-a856-01f4819ea710 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:10.474140 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac42c31-82e8-43dd-a856-01f4819ea710 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:10.483362 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:10.487001 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1248/2521] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:08:10.491828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad6ec4-0427-48a7-9a7b-1c611b99936d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:10.501460 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad6ec4-0427-48a7-9a7b-1c611b99936d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:10.506248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7192f3b-b283-450f-aede-4393ca986c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:10.506933 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7192f3b-b283-450f-aede-4393ca986c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:10.507314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7192f3b-b283-450f-aede-4393ca986c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:10.507851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7192f3b-b283-450f-aede-4393ca986c40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:10.508619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7192f3b-b283-450f-aede-4393ca986c40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:10.510528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7192f3b-b283-450f-aede-4393ca986c40 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:10.540540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1274/2522] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:08:10 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:08:10.545007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac42c31-82e8-43dd-a856-01f4819ea710 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:10.555385 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac42c31-82e8-43dd-a856-01f4819ea710 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:10.618038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b197b19-8c70-4344-8f2c-bb2b07ece6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:10.618652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b197b19-8c70-4344-8f2c-bb2b07ece6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:10.618930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b197b19-8c70-4344-8f2c-bb2b07ece6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:10.619197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b197b19-8c70-4344-8f2c-bb2b07ece6b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:10.619831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b197b19-8c70-4344-8f2c-bb2b07ece6b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:10.621942 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b197b19-8c70-4344-8f2c-bb2b07ece6b5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:10.645468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1249/2523] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:08:10 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:08:10.651583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7192f3b-b283-450f-aede-4393ca986c40 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:10.663310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7192f3b-b283-450f-aede-4393ca986c40 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:10.667903 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06a8c1-f897-4874-bf1a-6c7076f17ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:10.668543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06a8c1-f897-4874-bf1a-6c7076f17ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:10.668819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06a8c1-f897-4874-bf1a-6c7076f17ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:10.669087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06a8c1-f897-4874-bf1a-6c7076f17ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:10.669729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06a8c1-f897-4874-bf1a-6c7076f17ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:10.671349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06a8c1-f897-4874-bf1a-6c7076f17ff7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:10.682530 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1275/2524] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:08:10.686629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b197b19-8c70-4344-8f2c-bb2b07ece6b5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:10.698142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b197b19-8c70-4344-8f2c-bb2b07ece6b5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:10.703030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b5d07-67f7-4c78-ae0b-d4dfd6a05b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:10.703631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b5d07-67f7-4c78-ae0b-d4dfd6a05b26 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:10.703912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b5d07-67f7-4c78-ae0b-d4dfd6a05b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:10.704219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b5d07-67f7-4c78-ae0b-d4dfd6a05b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:10.705113 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b5d07-67f7-4c78-ae0b-d4dfd6a05b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:10.706725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b5d07-67f7-4c78-ae0b-d4dfd6a05b26 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:10.718495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1250/2525] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:08:10.722965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06a8c1-f897-4874-bf1a-6c7076f17ff7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:10.733048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06a8c1-f897-4874-bf1a-6c7076f17ff7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:10.738362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffe6db-c55b-4ae6-ad98-c0986c72b708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:10.739020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffe6db-c55b-4ae6-ad98-c0986c72b708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:10.739302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffe6db-c55b-4ae6-ad98-c0986c72b708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:10.739577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffe6db-c55b-4ae6-ad98-c0986c72b708 admin admin] PATH_INFO: `/v3/projects/dce02510fa6d40c4aefe75816bfd50d6/users/d3892b6f907143e9b2f16767d835c8ca/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:10.756269 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffe6db-c55b-4ae6-ad98-c0986c72b708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce02510fa6d40c4aefe75816bfd50d6, user_id=d3892b6f907143e9b2f16767d835c8ca, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:10.758986 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffe6db-c55b-4ae6-ad98-c0986c72b708 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:10.774168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1276/2526] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:08:10 2022] PUT /identity/v3/projects/dce02510fa6d40c4aefe75816bfd50d6/users/d3892b6f907143e9b2f16767d835c8ca/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:08:10.785686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356f57e-36ca-427f-aad3-19a03b41b733 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:10.786078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356f57e-36ca-427f-aad3-19a03b41b733 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:10.786425 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356f57e-36ca-427f-aad3-19a03b41b733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:10.820873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7356f57e-36ca-427f-aad3-19a03b41b733 None None] MFA Rules not processed for user `d3892b6f907143e9b2f16767d835c8ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:10.822101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7356f57e-36ca-427f-aad3-19a03b41b733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:10.899719 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1251/2527] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:08:10.912026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffe6db-c55b-4ae6-ad98-c0986c72b708 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:10.914013 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ebffe6db-c55b-4ae6-ad98-c0986c72b708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:10.972946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffe6db-c55b-4ae6-ad98-c0986c72b708 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:10.976834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f3bbf-1229-418a-b83e-1167febc63af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:10.977524 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f3bbf-1229-418a-b83e-1167febc63af service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:10.977882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f3bbf-1229-418a-b83e-1167febc63af service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:10.978225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f3bbf-1229-418a-b83e-1167febc63af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:10.983828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:10.989331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f3bbf-1229-418a-b83e-1167febc63af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:10.991118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f3bbf-1229-418a-b83e-1167febc63af service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:11.002784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1277/2528] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:08:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:08:20.896662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:20.998200 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:34.954760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356f57e-36ca-427f-aad3-19a03b41b733 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:35.023423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356f57e-36ca-427f-aad3-19a03b41b733 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:35.027933 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:35.028578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:35.028857 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:35.029126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] PATH_INFO: `/v3/users/d3892b6f907143e9b2f16767d835c8ca` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:35.030231 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3892b6f907143e9b2f16767d835c8ca)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:35.033108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:35.080230 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3892b6f907143e9b2f16767d835c8ca'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:35.087156 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3892b6f907143e9b2f16767d835c8ca'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:35.099966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3892b6f907143e9b2f16767d835c8ca'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:35.100687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3892b6f907143e9b2f16767d835c8ca'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:35.101315 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3892b6f907143e9b2f16767d835c8ca'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:35.115413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1252/2529] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:08:34 2022] DELETE /identity/v3/users/d3892b6f907143e9b2f16767d835c8ca => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:35.119791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba72b7-6fe8-4e21-bbd6-a8e4a7e6d564 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:35.120340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba72b7-6fe8-4e21-bbd6-a8e4a7e6d564 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:35.120462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba72b7-6fe8-4e21-bbd6-a8e4a7e6d564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:35.142916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-49ba72b7-6fe8-4e21-bbd6-a8e4a7e6d564 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:08:35.144169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-49ba72b7-6fe8-4e21-bbd6-a8e4a7e6d564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:35.219913 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1278/2530] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:08:35.229631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:35.231075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:35.291205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60717d7f-0b49-49c2-8529-ccbce0d355a4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:35.295027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d03923-c085-4554-ae0a-56175ac7ba4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:35.295640 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d03923-c085-4554-ae0a-56175ac7ba4a service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:35.295920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d03923-c085-4554-ae0a-56175ac7ba4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:35.296224 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d03923-c085-4554-ae0a-56175ac7ba4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:35.305583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d03923-c085-4554-ae0a-56175ac7ba4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:35.307051 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d03923-c085-4554-ae0a-56175ac7ba4a service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:35.318549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1253/2531] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:08:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:08:35.609538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba72b7-6fe8-4e21-bbd6-a8e4a7e6d564 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:08:35.611911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-49ba72b7-6fe8-4e21-bbd6-a8e4a7e6d564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:08:35.629521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba72b7-6fe8-4e21-bbd6-a8e4a7e6d564 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:08:35.635868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:08:35.637801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:08:35.638193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:08:35.638704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] PATH_INFO: `/v3/projects/dce02510fa6d40c4aefe75816bfd50d6` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:08:35.640471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce02510fa6d40c4aefe75816bfd50d6)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:08:35.643841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:08:35.694445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce02510fa6d40c4aefe75816bfd50d6', 'request_id': 'req-dcfc55d0-eeca-4f91-ae5c-2288790869a2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:35.700183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce02510fa6d40c4aefe75816bfd50d6', 'request_id': 'req-dcfc55d0-eeca-4f91-ae5c-2288790869a2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:35.707254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce02510fa6d40c4aefe75816bfd50d6', 'request_id': 'req-dcfc55d0-eeca-4f91-ae5c-2288790869a2'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:35.720864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] The token cache is being invalidate because project dce02510fa6d40c4aefe75816bfd50d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:08:35.721155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:35.721791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:08:35.723283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1279/2532] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:08:35 2022] DELETE /identity/v3/projects/dce02510fa6d40c4aefe75816bfd50d6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:08:45.312652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:08:45.717895 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:03.124564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d03923-c085-4554-ae0a-56175ac7ba4a service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:03.126012 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-18d03923-c085-4554-ae0a-56175ac7ba4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:03.212608 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d03923-c085-4554-ae0a-56175ac7ba4a service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:03.279449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbdcd3-bf88-426a-8f1a-c1f7faada60c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:03.280226 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbdcd3-bf88-426a-8f1a-c1f7faada60c service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:03.280576 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbdcd3-bf88-426a-8f1a-c1f7faada60c service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:03.280910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbdcd3-bf88-426a-8f1a-c1f7faada60c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:03.282701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-22bbdcd3-bf88-426a-8f1a-c1f7faada60c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:03.342859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbdcd3-bf88-426a-8f1a-c1f7faada60c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:03.344484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbdcd3-bf88-426a-8f1a-c1f7faada60c service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:03.354289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1254/2533] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:08.387625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:08.389075 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:08.458018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc55d0-eeca-4f91-ae5c-2288790869a2 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:08.461566 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37637872-9d2b-40b3-9334-260baa1cbc1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:08.462177 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37637872-9d2b-40b3-9334-260baa1cbc1a service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:08.462461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37637872-9d2b-40b3-9334-260baa1cbc1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:08.462736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37637872-9d2b-40b3-9334-260baa1cbc1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:08.464488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-37637872-9d2b-40b3-9334-260baa1cbc1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:08.553401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37637872-9d2b-40b3-9334-260baa1cbc1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:08.554757 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37637872-9d2b-40b3-9334-260baa1cbc1a service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:08.563664 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1280/2534] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:08.913008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbdcd3-bf88-426a-8f1a-c1f7faada60c service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:08.914465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-22bbdcd3-bf88-426a-8f1a-c1f7faada60c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:08.963637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:08.998041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbdcd3-bf88-426a-8f1a-c1f7faada60c service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:09.001723 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818c232-b79d-4595-ad70-3c37bbc9a57d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:09.002329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818c232-b79d-4595-ad70-3c37bbc9a57d service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:09.002651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818c232-b79d-4595-ad70-3c37bbc9a57d service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:09.002889 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818c232-b79d-4595-ad70-3c37bbc9a57d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:09.004653 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f818c232-b79d-4595-ad70-3c37bbc9a57d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:09.025988 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818c232-b79d-4595-ad70-3c37bbc9a57d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:09.027442 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818c232-b79d-4595-ad70-3c37bbc9a57d service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:09.039568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1255/2535] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:16.822488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37637872-9d2b-40b3-9334-260baa1cbc1a service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:16.833398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-37637872-9d2b-40b3-9334-260baa1cbc1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:16.950909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818c232-b79d-4595-ad70-3c37bbc9a57d service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:16.962247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37637872-9d2b-40b3-9334-260baa1cbc1a service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:16.966029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818c232-b79d-4595-ad70-3c37bbc9a57d service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:16.968773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c4b79-8c11-4234-8382-309d70c6a3dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:16.969698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c4b79-8c11-4234-8382-309d70c6a3dd service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:16.970047 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c4b79-8c11-4234-8382-309d70c6a3dd service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:16.970384 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c4b79-8c11-4234-8382-309d70c6a3dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:16.972771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1a2c4b79-8c11-4234-8382-309d70c6a3dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:16.976506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9b305-6a34-4605-a5d1-b7b3893db2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:16.977127 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9b305-6a34-4605-a5d1-b7b3893db2eb service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:16.977406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9b305-6a34-4605-a5d1-b7b3893db2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:16.977679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9b305-6a34-4605-a5d1-b7b3893db2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:16.985270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f5f9b305-6a34-4605-a5d1-b7b3893db2eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:16.990190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c4b79-8c11-4234-8382-309d70c6a3dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:16.992300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c4b79-8c11-4234-8382-309d70c6a3dd service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:17.011965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1281/2536] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:17.114948 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9b305-6a34-4605-a5d1-b7b3893db2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:17.116811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9b305-6a34-4605-a5d1-b7b3893db2eb service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:17.132587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1256/2537] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:21.433830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c4b79-8c11-4234-8382-309d70c6a3dd service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:21.438319 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1a2c4b79-8c11-4234-8382-309d70c6a3dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:21.456247 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:21.463074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c4b79-8c11-4234-8382-309d70c6a3dd service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:21.467644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c32702-f424-47d7-a01c-241c34693312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:21.468622 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c32702-f424-47d7-a01c-241c34693312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:21.468910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c32702-f424-47d7-a01c-241c34693312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:21.469197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c32702-f424-47d7-a01c-241c34693312 admin admin] PATH_INFO: `/v3/users/0d1dbe40cb994491bafc6ce1f947f4f6` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:21.470331 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c32702-f424-47d7-a01c-241c34693312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d1dbe40cb994491bafc6ce1f947f4f6)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:21.473585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c32702-f424-47d7-a01c-241c34693312 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:21.543753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c7c32702-f424-47d7-a01c-241c34693312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d1dbe40cb994491bafc6ce1f947f4f6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:21.544513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c7c32702-f424-47d7-a01c-241c34693312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d1dbe40cb994491bafc6ce1f947f4f6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:21.553992 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c7c32702-f424-47d7-a01c-241c34693312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d1dbe40cb994491bafc6ce1f947f4f6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:21.563787 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c7c32702-f424-47d7-a01c-241c34693312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d1dbe40cb994491bafc6ce1f947f4f6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:21.581675 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c7c32702-f424-47d7-a01c-241c34693312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d1dbe40cb994491bafc6ce1f947f4f6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:21.596321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1282/2538] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:21 2022] DELETE /identity/v3/users/0d1dbe40cb994491bafc6ce1f947f4f6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:21.798433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9b305-6a34-4605-a5d1-b7b3893db2eb service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:21.811330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9b305-6a34-4605-a5d1-b7b3893db2eb service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:21.898532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aff224-3814-4cd5-a16b-7cade1902841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:21.899585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aff224-3814-4cd5-a16b-7cade1902841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:21.899877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aff224-3814-4cd5-a16b-7cade1902841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:21.900205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aff224-3814-4cd5-a16b-7cade1902841 admin admin] PATH_INFO: `/v3/projects/37c662b75f1d46619c3993132d413549` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:21.901784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aff224-3814-4cd5-a16b-7cade1902841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c662b75f1d46619c3993132d413549)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:21.904091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aff224-3814-4cd5-a16b-7cade1902841 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:21.980745 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a9aff224-3814-4cd5-a16b-7cade1902841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c662b75f1d46619c3993132d413549', 'request_id': 'req-a9aff224-3814-4cd5-a16b-7cade1902841'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:21.989520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a9aff224-3814-4cd5-a16b-7cade1902841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c662b75f1d46619c3993132d413549', 'request_id': 'req-a9aff224-3814-4cd5-a16b-7cade1902841'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:21.992280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:21.998898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a9aff224-3814-4cd5-a16b-7cade1902841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c662b75f1d46619c3993132d413549', 'request_id': 'req-a9aff224-3814-4cd5-a16b-7cade1902841'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:22.012585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a9aff224-3814-4cd5-a16b-7cade1902841 admin admin] The token cache is being invalidate because project 37c662b75f1d46619c3993132d413549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:22.012704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a9aff224-3814-4cd5-a16b-7cade1902841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:22.013398 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a9aff224-3814-4cd5-a16b-7cade1902841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:22.014873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1257/2539] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:21 2022] DELETE /identity/v3/projects/37c662b75f1d46619c3993132d413549 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:22.019978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4fa2d-d661-4fe9-b38e-eafb34aae26b None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:22.020316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4fa2d-d661-4fe9-b38e-eafb34aae26b None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:22.020590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4fa2d-d661-4fe9-b38e-eafb34aae26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:22.043560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-11c4fa2d-d661-4fe9-b38e-eafb34aae26b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:22.044784 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-11c4fa2d-d661-4fe9-b38e-eafb34aae26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:22.107938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1283/2540] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:22.177063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f04919-acd3-49d7-83b0-2890586a6c9a None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:22.177333 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f04919-acd3-49d7-83b0-2890586a6c9a None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:22.177599 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f04919-acd3-49d7-83b0-2890586a6c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:22.197700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e1f04919-acd3-49d7-83b0-2890586a6c9a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:22.198924 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e1f04919-acd3-49d7-83b0-2890586a6c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:22.206936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1258/2541] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:22.211170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4fa2d-d661-4fe9-b38e-eafb34aae26b None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:22.222018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4fa2d-d661-4fe9-b38e-eafb34aae26b None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:22.228278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc16b4e-1335-47c9-97fb-cf0e2a51d9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:22.228903 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc16b4e-1335-47c9-97fb-cf0e2a51d9c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:22.229185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc16b4e-1335-47c9-97fb-cf0e2a51d9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:22.229456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc16b4e-1335-47c9-97fb-cf0e2a51d9c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:22.230108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc16b4e-1335-47c9-97fb-cf0e2a51d9c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:09:22.230428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc16b4e-1335-47c9-97fb-cf0e2a51d9c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:22.232040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc16b4e-1335-47c9-97fb-cf0e2a51d9c4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:22.246588 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1284/2542] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:09:22.251132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f04919-acd3-49d7-83b0-2890586a6c9a None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:22.260541 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f04919-acd3-49d7-83b0-2890586a6c9a None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:22.265100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9e22d-920b-48f9-84f8-1d9ca6221543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:22.265702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9e22d-920b-48f9-84f8-1d9ca6221543 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:22.265959 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9e22d-920b-48f9-84f8-1d9ca6221543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:22.266227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9e22d-920b-48f9-84f8-1d9ca6221543 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:22.266852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9e22d-920b-48f9-84f8-1d9ca6221543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:22.269126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df9e22d-920b-48f9-84f8-1d9ca6221543 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:22.297762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1259/2543] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:22 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:22.303156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc16b4e-1335-47c9-97fb-cf0e2a51d9c4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:22.313047 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc16b4e-1335-47c9-97fb-cf0e2a51d9c4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:22.373637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a180ce-7780-4c5e-b5b3-0e4f465d0e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:22.374159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a180ce-7780-4c5e-b5b3-0e4f465d0e4a admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:22.374437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a180ce-7780-4c5e-b5b3-0e4f465d0e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:22.374705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a180ce-7780-4c5e-b5b3-0e4f465d0e4a admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:22.375329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a180ce-7780-4c5e-b5b3-0e4f465d0e4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:22.377431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a180ce-7780-4c5e-b5b3-0e4f465d0e4a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:22.386696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:22.401804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1285/2544] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:22 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:22.406085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9e22d-920b-48f9-84f8-1d9ca6221543 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:22.417608 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9e22d-920b-48f9-84f8-1d9ca6221543 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:22.422551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50cd86-a3c1-47e8-a903-8cd4a14ecea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:22.423146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50cd86-a3c1-47e8-a903-8cd4a14ecea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:22.423424 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50cd86-a3c1-47e8-a903-8cd4a14ecea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:22.423693 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50cd86-a3c1-47e8-a903-8cd4a14ecea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:22.425037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50cd86-a3c1-47e8-a903-8cd4a14ecea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:22.426664 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50cd86-a3c1-47e8-a903-8cd4a14ecea2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:22.436749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1260/2545] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:22.441258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a180ce-7780-4c5e-b5b3-0e4f465d0e4a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:22.451284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a180ce-7780-4c5e-b5b3-0e4f465d0e4a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:22.456906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c50f14-e015-40ab-958b-9882c4e189b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:22.457513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c50f14-e015-40ab-958b-9882c4e189b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:22.457968 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c50f14-e015-40ab-958b-9882c4e189b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:22.458244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c50f14-e015-40ab-958b-9882c4e189b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:22.458912 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c50f14-e015-40ab-958b-9882c4e189b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:22.460536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c50f14-e015-40ab-958b-9882c4e189b2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:22.473559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1286/2546] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:22.477843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50cd86-a3c1-47e8-a903-8cd4a14ecea2 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:22.488042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50cd86-a3c1-47e8-a903-8cd4a14ecea2 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:22.492573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04ac46-0455-4b63-9ee8-81664aab76a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:22.493210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04ac46-0455-4b63-9ee8-81664aab76a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:22.493506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04ac46-0455-4b63-9ee8-81664aab76a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:22.493781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04ac46-0455-4b63-9ee8-81664aab76a7 admin admin] PATH_INFO: `/v3/projects/5846c04e851a48499c81895b21391064/users/e2eea21de17041a890bddf77e4e519d4/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:22.508518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04ac46-0455-4b63-9ee8-81664aab76a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5846c04e851a48499c81895b21391064, user_id=e2eea21de17041a890bddf77e4e519d4, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:22.511160 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04ac46-0455-4b63-9ee8-81664aab76a7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:22.525175 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1261/2547] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:22 2022] PUT /identity/v3/projects/5846c04e851a48499c81895b21391064/users/e2eea21de17041a890bddf77e4e519d4/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:22.535746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4aba9-f609-4eb4-a6c9-b0acec754af5 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:22.536064 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4aba9-f609-4eb4-a6c9-b0acec754af5 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:22.536356 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4aba9-f609-4eb4-a6c9-b0acec754af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:22.565318 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-51a4aba9-f609-4eb4-a6c9-b0acec754af5 None None] MFA Rules not processed for user `e2eea21de17041a890bddf77e4e519d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:22.566543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-51a4aba9-f609-4eb4-a6c9-b0acec754af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:22.632893 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1287/2548] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:22.645099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04ac46-0455-4b63-9ee8-81664aab76a7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:22.647811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1b04ac46-0455-4b63-9ee8-81664aab76a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:22.731624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04ac46-0455-4b63-9ee8-81664aab76a7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:22.739860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae2108-9750-4de6-a241-3123d5186d6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:22.739860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae2108-9750-4de6-a241-3123d5186d6e service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:22.739860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae2108-9750-4de6-a241-3123d5186d6e service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:22.739860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae2108-9750-4de6-a241-3123d5186d6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:22.748752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae2108-9750-4de6-a241-3123d5186d6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:22.750371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae2108-9750-4de6-a241-3123d5186d6e service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:22.761506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1262/2549] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:24.040558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4aba9-f609-4eb4-a6c9-b0acec754af5 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:24.042006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-51a4aba9-f609-4eb4-a6c9-b0acec754af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:24.103855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4aba9-f609-4eb4-a6c9-b0acec754af5 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:24.107434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ff005-bda3-441b-9240-16fb3fe560f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:24.108083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ff005-bda3-441b-9240-16fb3fe560f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:24.108364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ff005-bda3-441b-9240-16fb3fe560f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:24.108636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ff005-bda3-441b-9240-16fb3fe560f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:24.110470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d86ff005-bda3-441b-9240-16fb3fe560f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:24.174488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ff005-bda3-441b-9240-16fb3fe560f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:24.176253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ff005-bda3-441b-9240-16fb3fe560f1 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:24.186300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1288/2550] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:25.539166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae2108-9750-4de6-a241-3123d5186d6e service nova] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:25.548115 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae2108-9750-4de6-a241-3123d5186d6e service nova] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:25.553825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:25.554725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:25.555080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:25.555429 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] PATH_INFO: `/v3/users/e2eea21de17041a890bddf77e4e519d4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:25.556673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2eea21de17041a890bddf77e4e519d4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:25.558700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:25.632410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2eea21de17041a890bddf77e4e519d4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:25.642953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2eea21de17041a890bddf77e4e519d4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:25.651991 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:25.681337 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2eea21de17041a890bddf77e4e519d4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:25.682549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2eea21de17041a890bddf77e4e519d4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:25.683174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2eea21de17041a890bddf77e4e519d4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:25.713924 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1263/2551] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:25 2022] DELETE /identity/v3/users/e2eea21de17041a890bddf77e4e519d4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:25.718408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381660de-204a-435f-9e69-b870cfe99f40 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:25.719280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381660de-204a-435f-9e69-b870cfe99f40 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:25.719710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381660de-204a-435f-9e69-b870cfe99f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:25.752548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-381660de-204a-435f-9e69-b870cfe99f40 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:25.754233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-381660de-204a-435f-9e69-b870cfe99f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:25.769307 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:25.837346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1289/2552] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:25.848443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:25.850895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:25.921057 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fb89a-3980-4488-b7a9-c4829a369aa6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:25.924947 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682319f-f54d-435b-8c74-def98ccbf158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:25.925642 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682319f-f54d-435b-8c74-def98ccbf158 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:25.926100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682319f-f54d-435b-8c74-def98ccbf158 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:25.926445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682319f-f54d-435b-8c74-def98ccbf158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:25.936988 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682319f-f54d-435b-8c74-def98ccbf158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:25.938685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682319f-f54d-435b-8c74-def98ccbf158 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:25.950994 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1264/2553] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:26.368443 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381660de-204a-435f-9e69-b870cfe99f40 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:26.369874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-381660de-204a-435f-9e69-b870cfe99f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:26.382284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381660de-204a-435f-9e69-b870cfe99f40 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:26.388480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b76674-9fe4-42f1-9b7c-30ca1baff845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:26.389581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b76674-9fe4-42f1-9b7c-30ca1baff845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:26.390090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b76674-9fe4-42f1-9b7c-30ca1baff845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:26.390587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b76674-9fe4-42f1-9b7c-30ca1baff845 admin admin] PATH_INFO: `/v3/projects/5846c04e851a48499c81895b21391064` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:26.392309 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b76674-9fe4-42f1-9b7c-30ca1baff845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5846c04e851a48499c81895b21391064)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:26.396495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b76674-9fe4-42f1-9b7c-30ca1baff845 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:26.450771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f1b76674-9fe4-42f1-9b7c-30ca1baff845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5846c04e851a48499c81895b21391064', 'request_id': 'req-f1b76674-9fe4-42f1-9b7c-30ca1baff845'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:26.459599 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f1b76674-9fe4-42f1-9b7c-30ca1baff845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5846c04e851a48499c81895b21391064', 'request_id': 'req-f1b76674-9fe4-42f1-9b7c-30ca1baff845'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:26.467819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f1b76674-9fe4-42f1-9b7c-30ca1baff845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5846c04e851a48499c81895b21391064', 'request_id': 'req-f1b76674-9fe4-42f1-9b7c-30ca1baff845'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:26.485280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f1b76674-9fe4-42f1-9b7c-30ca1baff845 admin admin] The token cache is being invalidate because project 5846c04e851a48499c81895b21391064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:26.485938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f1b76674-9fe4-42f1-9b7c-30ca1baff845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:26.487544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f1b76674-9fe4-42f1-9b7c-30ca1baff845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:26.489345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1290/2554] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:26 2022] DELETE /identity/v3/projects/5846c04e851a48499c81895b21391064 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:26.494442 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360ca4-8f6c-4fdd-902a-2d88cfb38413 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:26.494966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360ca4-8f6c-4fdd-902a-2d88cfb38413 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:26.495451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360ca4-8f6c-4fdd-902a-2d88cfb38413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:26.516461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-39360ca4-8f6c-4fdd-902a-2d88cfb38413 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:26.517898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-39360ca4-8f6c-4fdd-902a-2d88cfb38413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:26.600433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1265/2555] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:26.604528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777a6ee-5618-4f75-8ce9-c72900aa81de None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:26.610192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777a6ee-5618-4f75-8ce9-c72900aa81de None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:26.610487 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777a6ee-5618-4f75-8ce9-c72900aa81de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:26.651359 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f777a6ee-5618-4f75-8ce9-c72900aa81de None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:26.653144 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f777a6ee-5618-4f75-8ce9-c72900aa81de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:26.664781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1291/2556] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:26.668927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39360ca4-8f6c-4fdd-902a-2d88cfb38413 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:26.679804 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39360ca4-8f6c-4fdd-902a-2d88cfb38413 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:26.684749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d1629-f254-4555-97f7-9f872c5d58c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:26.685495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d1629-f254-4555-97f7-9f872c5d58c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:26.685888 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d1629-f254-4555-97f7-9f872c5d58c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:26.686251 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d1629-f254-4555-97f7-9f872c5d58c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:26.687010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d1629-f254-4555-97f7-9f872c5d58c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:09:26.687420 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d1629-f254-4555-97f7-9f872c5d58c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:26.689849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d1629-f254-4555-97f7-9f872c5d58c9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:26.711043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1266/2557] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:09:26.711655 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777a6ee-5618-4f75-8ce9-c72900aa81de None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:26.721994 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777a6ee-5618-4f75-8ce9-c72900aa81de None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:26.727101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f29432-890c-4749-8f18-b40adf540dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:26.727490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f29432-890c-4749-8f18-b40adf540dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:26.727765 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f29432-890c-4749-8f18-b40adf540dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:26.728102 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f29432-890c-4749-8f18-b40adf540dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:26.728738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f29432-890c-4749-8f18-b40adf540dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:26.730899 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f29432-890c-4749-8f18-b40adf540dd9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:26.770870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1292/2558] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:26 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:26.772435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d1629-f254-4555-97f7-9f872c5d58c9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:26.783136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d1629-f254-4555-97f7-9f872c5d58c9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:26.878704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd941ccb-d5db-416d-b93c-624410371fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:26.879408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd941ccb-d5db-416d-b93c-624410371fac admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:26.879822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd941ccb-d5db-416d-b93c-624410371fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:26.880138 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd941ccb-d5db-416d-b93c-624410371fac admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:26.880853 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd941ccb-d5db-416d-b93c-624410371fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:26.882951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd941ccb-d5db-416d-b93c-624410371fac admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:26.904108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:26.912463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1267/2559] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:26 2022] POST /identity/v3/users => generated 303 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:26.915788 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f29432-890c-4749-8f18-b40adf540dd9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:26.927685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f29432-890c-4749-8f18-b40adf540dd9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:26.932726 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543fa51-0561-41e8-8ed0-7906767050b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:26.933777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543fa51-0561-41e8-8ed0-7906767050b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:26.933777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543fa51-0561-41e8-8ed0-7906767050b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:26.933974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543fa51-0561-41e8-8ed0-7906767050b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:26.934537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543fa51-0561-41e8-8ed0-7906767050b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:26.936173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543fa51-0561-41e8-8ed0-7906767050b7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:26.951172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1293/2560] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:26.952339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd941ccb-d5db-416d-b93c-624410371fac admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:26.963406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd941ccb-d5db-416d-b93c-624410371fac admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:26.968330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c522a-27ee-4f02-b9b8-7ad0b26fafcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:26.968937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c522a-27ee-4f02-b9b8-7ad0b26fafcb admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:26.969219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c522a-27ee-4f02-b9b8-7ad0b26fafcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:26.969491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c522a-27ee-4f02-b9b8-7ad0b26fafcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:26.970164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c522a-27ee-4f02-b9b8-7ad0b26fafcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:26.972364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c522a-27ee-4f02-b9b8-7ad0b26fafcb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:26.987940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1268/2561] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:09:26.991618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543fa51-0561-41e8-8ed0-7906767050b7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:27.001264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543fa51-0561-41e8-8ed0-7906767050b7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:27.006336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372791a6-b865-4806-b73e-e7d4f7d8b862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:27.007035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372791a6-b865-4806-b73e-e7d4f7d8b862 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:27.007276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372791a6-b865-4806-b73e-e7d4f7d8b862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:27.007545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372791a6-b865-4806-b73e-e7d4f7d8b862 admin admin] PATH_INFO: `/v3/projects/3116b0dd174541268ee6d2a7605f615c/users/6ac5add42b8846aa87cf5e115db6fa3a/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:27.028498 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372791a6-b865-4806-b73e-e7d4f7d8b862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3116b0dd174541268ee6d2a7605f615c, user_id=6ac5add42b8846aa87cf5e115db6fa3a, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:27.031077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372791a6-b865-4806-b73e-e7d4f7d8b862 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:27.057074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1294/2562] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:26 2022] PUT /identity/v3/projects/3116b0dd174541268ee6d2a7605f615c/users/6ac5add42b8846aa87cf5e115db6fa3a/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:27.059685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179aa890-582f-4bc6-8643-6b2f4960e050 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:27.059685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179aa890-582f-4bc6-8643-6b2f4960e050 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:27.059685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179aa890-582f-4bc6-8643-6b2f4960e050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:27.089583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-179aa890-582f-4bc6-8643-6b2f4960e050 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:27.090898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-179aa890-582f-4bc6-8643-6b2f4960e050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:27.176041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1269/2563] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:09:27.184217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372791a6-b865-4806-b73e-e7d4f7d8b862 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:27.186251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-372791a6-b865-4806-b73e-e7d4f7d8b862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:27.270605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:27.307486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372791a6-b865-4806-b73e-e7d4f7d8b862 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:27.311343 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ee1d0-c4de-4e27-a885-b9420cb1c562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:27.311961 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ee1d0-c4de-4e27-a885-b9420cb1c562 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:27.312290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ee1d0-c4de-4e27-a885-b9420cb1c562 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:27.312564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ee1d0-c4de-4e27-a885-b9420cb1c562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:27.333174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ee1d0-c4de-4e27-a885-b9420cb1c562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:27.335381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ee1d0-c4de-4e27-a885-b9420cb1c562 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:27.357093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1295/2564] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:27.962814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179aa890-582f-4bc6-8643-6b2f4960e050 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:27.964370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-179aa890-582f-4bc6-8643-6b2f4960e050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:27.985025 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179aa890-582f-4bc6-8643-6b2f4960e050 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:27.992190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:27.993046 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:27.993487 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:27.994001 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] PATH_INFO: `/v3/users/54b72a430d9d4d43954eb56d703079b4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:27.995442 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54b72a430d9d4d43954eb56d703079b4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:27.997894 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:28.070441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54b72a430d9d4d43954eb56d703079b4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:28.076885 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54b72a430d9d4d43954eb56d703079b4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:28.092269 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54b72a430d9d4d43954eb56d703079b4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:28.093092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54b72a430d9d4d43954eb56d703079b4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:28.093762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54b72a430d9d4d43954eb56d703079b4'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:28.115728 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1270/2565] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:27 2022] DELETE /identity/v3/users/54b72a430d9d4d43954eb56d703079b4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:28.335014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a65b48-e4e2-46e2-b00c-f8c53252e360 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:28.335490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a65b48-e4e2-46e2-b00c-f8c53252e360 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:28.335571 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a65b48-e4e2-46e2-b00c-f8c53252e360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:28.402933 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:28.406284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:28.418182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f1a65b48-e4e2-46e2-b00c-f8c53252e360 None None] MFA Rules not processed for user `6ac5add42b8846aa87cf5e115db6fa3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:28.419635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f1a65b48-e4e2-46e2-b00c-f8c53252e360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:28.509634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1296/2566] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:28 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:28.516897 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cdf6c-4f62-4991-84b5-178adb67ed09 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:28.522462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:28.523555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:28.523940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:28.524377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3 admin admin] PATH_INFO: `/v3/projects/0a79d2a9724e49a9934d9ca0184b31fd` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:28.524846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a65b48-e4e2-46e2-b00c-f8c53252e360 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:28.526002 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a79d2a9724e49a9934d9ca0184b31fd)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:28.527245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f1a65b48-e4e2-46e2-b00c-f8c53252e360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:28.528288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:28.531275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:28.545569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a65b48-e4e2-46e2-b00c-f8c53252e360 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:28.551396 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446392c1-4f86-4ce3-a21f-d0df02583fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:28.552087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446392c1-4f86-4ce3-a21f-d0df02583fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:28.552410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446392c1-4f86-4ce3-a21f-d0df02583fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:28.552687 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446392c1-4f86-4ce3-a21f-d0df02583fc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:28.553311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446392c1-4f86-4ce3-a21f-d0df02583fc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:28.555203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446392c1-4f86-4ce3-a21f-d0df02583fc1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:28.586141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1297/2567] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:28 2022] POST /identity/v3/projects => generated 405 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:28.588452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a79d2a9724e49a9934d9ca0184b31fd', 'request_id': 'req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:28.590934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446392c1-4f86-4ce3-a21f-d0df02583fc1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:28.598522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a79d2a9724e49a9934d9ca0184b31fd', 'request_id': 'req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:28.606562 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a79d2a9724e49a9934d9ca0184b31fd', 'request_id': 'req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:28.619831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3 admin admin] The token cache is being invalidate because project 0a79d2a9724e49a9934d9ca0184b31fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:28.620247 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:28.621121 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-16e9f1c3-fbc3-44b7-ae3b-2bb32a98dff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:28.622780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1271/2568] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:28 2022] DELETE /identity/v3/projects/0a79d2a9724e49a9934d9ca0184b31fd => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:28.627526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16761417-7242-44c7-8c8d-ff01c398a2a2 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:28.627907 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16761417-7242-44c7-8c8d-ff01c398a2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:28.628327 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16761417-7242-44c7-8c8d-ff01c398a2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:28.650800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-16761417-7242-44c7-8c8d-ff01c398a2a2 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:28.652156 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-16761417-7242-44c7-8c8d-ff01c398a2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:28.668729 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446392c1-4f86-4ce3-a21f-d0df02583fc1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:28.674537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a45dc-07b6-405e-a4fd-a5571a571436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:28.675332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a45dc-07b6-405e-a4fd-a5571a571436 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:28.675782 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a45dc-07b6-405e-a4fd-a5571a571436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:28.676234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a45dc-07b6-405e-a4fd-a5571a571436 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:28.677031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a45dc-07b6-405e-a4fd-a5571a571436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:28.679124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a45dc-07b6-405e-a4fd-a5571a571436 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:28.702078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1298/2569] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:28 2022] POST /identity/v3/users => generated 309 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:28.706826 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a45dc-07b6-405e-a4fd-a5571a571436 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:28.709091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-651a45dc-07b6-405e-a4fd-a5571a571436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:28.727215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1272/2570] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:28.727423 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a45dc-07b6-405e-a4fd-a5571a571436 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:28.727423 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c5999-9c49-44cd-922a-b1e59509b9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:28.727760 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c5999-9c49-44cd-922a-b1e59509b9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:28.728272 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c5999-9c49-44cd-922a-b1e59509b9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:28.728640 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c5999-9c49-44cd-922a-b1e59509b9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:28.728737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a92f1-ef4d-4f4d-8b5b-d884dcc0f520 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:28.728974 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a92f1-ef4d-4f4d-8b5b-d884dcc0f520 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:28.729252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a92f1-ef4d-4f4d-8b5b-d884dcc0f520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:28.729612 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c5999-9c49-44cd-922a-b1e59509b9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:28.731295 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c5999-9c49-44cd-922a-b1e59509b9a4 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:28.747777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1299/2571] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:09:28.748962 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c5999-9c49-44cd-922a-b1e59509b9a4 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:28.751710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-031a92f1-ef4d-4f4d-8b5b-d884dcc0f520 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:28.753511 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-031a92f1-ef4d-4f4d-8b5b-d884dcc0f520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:28.760047 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c5999-9c49-44cd-922a-b1e59509b9a4 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:28.761823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1273/2572] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:09:28.764760 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95649d5-9615-4c5f-bedd-fb3a03399d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:28.765433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95649d5-9615-4c5f-bedd-fb3a03399d7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:28.765713 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95649d5-9615-4c5f-bedd-fb3a03399d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:28.766009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95649d5-9615-4c5f-bedd-fb3a03399d7e admin admin] PATH_INFO: `/v3/projects/1cc131699be546789ea565bbe314a062/users/3f602ff23d6141bc9d18655986c3044e/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:28.769216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a92f1-ef4d-4f4d-8b5b-d884dcc0f520 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:28.781134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a92f1-ef4d-4f4d-8b5b-d884dcc0f520 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:28.783906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95649d5-9615-4c5f-bedd-fb3a03399d7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc131699be546789ea565bbe314a062, user_id=3f602ff23d6141bc9d18655986c3044e, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:28.787425 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95649d5-9615-4c5f-bedd-fb3a03399d7e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:28.787987 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05cfd4-b1ea-4397-95ad-ec6405161405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:28.793103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05cfd4-b1ea-4397-95ad-ec6405161405 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:28.793103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05cfd4-b1ea-4397-95ad-ec6405161405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:28.793103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05cfd4-b1ea-4397-95ad-ec6405161405 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:28.793103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05cfd4-b1ea-4397-95ad-ec6405161405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:09:28.793103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05cfd4-b1ea-4397-95ad-ec6405161405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:28.793596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05cfd4-b1ea-4397-95ad-ec6405161405 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:28.807278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1300/2573] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:28 2022] PUT /identity/v3/projects/1cc131699be546789ea565bbe314a062/users/3f602ff23d6141bc9d18655986c3044e/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:28.810704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1274/2574] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 06:09:28.811035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95649d5-9615-4c5f-bedd-fb3a03399d7e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:28.814816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05cfd4-b1ea-4397-95ad-ec6405161405 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:28.818224 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:28.826741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05cfd4-b1ea-4397-95ad-ec6405161405 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:28.882352 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95649d5-9615-4c5f-bedd-fb3a03399d7e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:28.888155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee86a0-9bb5-4b3d-a7d5-6a0e995089e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:28.888810 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee86a0-9bb5-4b3d-a7d5-6a0e995089e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:28.889093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee86a0-9bb5-4b3d-a7d5-6a0e995089e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:28.889369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee86a0-9bb5-4b3d-a7d5-6a0e995089e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:28.890023 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee86a0-9bb5-4b3d-a7d5-6a0e995089e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:28.891203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef3664-7d72-4851-82b0-d433caf38983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:28.891657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee86a0-9bb5-4b3d-a7d5-6a0e995089e5 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:28.891809 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef3664-7d72-4851-82b0-d433caf38983 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:28.892136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef3664-7d72-4851-82b0-d433caf38983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:28.892447 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef3664-7d72-4851-82b0-d433caf38983 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:28.893092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef3664-7d72-4851-82b0-d433caf38983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:28.895354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef3664-7d72-4851-82b0-d433caf38983 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:28.903050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1301/2575] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:09:28.907143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee86a0-9bb5-4b3d-a7d5-6a0e995089e5 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:28.917454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee86a0-9bb5-4b3d-a7d5-6a0e995089e5 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:28.917882 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:28.922516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43fc06-3c70-463a-a0b6-ed5544329fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:28.923129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43fc06-3c70-463a-a0b6-ed5544329fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:28.923432 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43fc06-3c70-463a-a0b6-ed5544329fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:28.923801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43fc06-3c70-463a-a0b6-ed5544329fc7 admin admin] PATH_INFO: `/v3/projects/1cc131699be546789ea565bbe314a062/users/3f602ff23d6141bc9d18655986c3044e/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:28.925885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43fc06-3c70-463a-a0b6-ed5544329fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc131699be546789ea565bbe314a062, user_id=3f602ff23d6141bc9d18655986c3044e, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:28.926944 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1275/2576] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:28 2022] POST /identity/v3/projects => generated 357 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:28.928779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43fc06-3c70-463a-a0b6-ed5544329fc7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:28.930743 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef3664-7d72-4851-82b0-d433caf38983 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:28.940233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1302/2577] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:28 2022] PUT /identity/v3/projects/1cc131699be546789ea565bbe314a062/users/3f602ff23d6141bc9d18655986c3044e/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:28.999761 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef3664-7d72-4851-82b0-d433caf38983 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:29.073657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf599e-3c5f-432b-b226-cf14c44d3561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:29.074279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf599e-3c5f-432b-b226-cf14c44d3561 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:29.074678 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf599e-3c5f-432b-b226-cf14c44d3561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:29.074971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf599e-3c5f-432b-b226-cf14c44d3561 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:29.075655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf599e-3c5f-432b-b226-cf14c44d3561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:29.077760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf599e-3c5f-432b-b226-cf14c44d3561 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:29.106794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1276/2578] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:28 2022] POST /identity/v3/users => generated 279 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:29.110913 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43fc06-3c70-463a-a0b6-ed5544329fc7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:29.130252 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43fc06-3c70-463a-a0b6-ed5544329fc7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:29.136856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95dfc7-5416-4a15-962a-f18e0f1cbe57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:29.137605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95dfc7-5416-4a15-962a-f18e0f1cbe57 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:29.138174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95dfc7-5416-4a15-962a-f18e0f1cbe57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:29.138544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95dfc7-5416-4a15-962a-f18e0f1cbe57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:29.139348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95dfc7-5416-4a15-962a-f18e0f1cbe57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:29.141213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95dfc7-5416-4a15-962a-f18e0f1cbe57 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:29.154488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1303/2579] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:09:29.158614 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf599e-3c5f-432b-b226-cf14c44d3561 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:29.170011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf599e-3c5f-432b-b226-cf14c44d3561 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:29.175850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd2ef3-1f81-46cd-94ec-33961e4f35ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:29.176570 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd2ef3-1f81-46cd-94ec-33961e4f35ae admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:29.176925 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd2ef3-1f81-46cd-94ec-33961e4f35ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:29.177272 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd2ef3-1f81-46cd-94ec-33961e4f35ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:29.178023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd2ef3-1f81-46cd-94ec-33961e4f35ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:29.179813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd2ef3-1f81-46cd-94ec-33961e4f35ae admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:29.191360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1277/2580] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:29.195554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95dfc7-5416-4a15-962a-f18e0f1cbe57 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:29.208678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95dfc7-5416-4a15-962a-f18e0f1cbe57 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:29.215041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4ccb0-5c6f-4a3e-8b89-0b21ab936b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:29.215764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4ccb0-5c6f-4a3e-8b89-0b21ab936b59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:29.216131 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4ccb0-5c6f-4a3e-8b89-0b21ab936b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:29.216482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4ccb0-5c6f-4a3e-8b89-0b21ab936b59 admin admin] PATH_INFO: `/v3/projects/8783ad64d8b34a1b9e899199ccb56ae8/users/b91f35a090a6450c90ee08ef8b121f7c/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:29.238337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4ccb0-5c6f-4a3e-8b89-0b21ab936b59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8783ad64d8b34a1b9e899199ccb56ae8, user_id=b91f35a090a6450c90ee08ef8b121f7c, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:29.241279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4ccb0-5c6f-4a3e-8b89-0b21ab936b59 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:29.262046 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1304/2581] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:29 2022] PUT /identity/v3/projects/8783ad64d8b34a1b9e899199ccb56ae8/users/b91f35a090a6450c90ee08ef8b121f7c/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:29.272898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927249d-4091-4442-8f01-8c1c1fe7e6bc None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:29.273383 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927249d-4091-4442-8f01-8c1c1fe7e6bc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:29.273658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927249d-4091-4442-8f01-8c1c1fe7e6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:29.323532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b927249d-4091-4442-8f01-8c1c1fe7e6bc None None] MFA Rules not processed for user `b91f35a090a6450c90ee08ef8b121f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:29.325082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b927249d-4091-4442-8f01-8c1c1fe7e6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:29.443174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1278/2582] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:29.443607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4ccb0-5c6f-4a3e-8b89-0b21ab936b59 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:29.515569 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4ccb0-5c6f-4a3e-8b89-0b21ab936b59 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:29.521676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a3c3c-b3db-433d-bcf0-19257970b560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:29.522703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a3c3c-b3db-433d-bcf0-19257970b560 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:29.522986 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a3c3c-b3db-433d-bcf0-19257970b560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:29.523258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a3c3c-b3db-433d-bcf0-19257970b560 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:29.524027 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a3c3c-b3db-433d-bcf0-19257970b560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:29.526073 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a3c3c-b3db-433d-bcf0-19257970b560 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:29.558461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1305/2583] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:29 2022] POST /identity/v3/projects => generated 355 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:29.562438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927249d-4091-4442-8f01-8c1c1fe7e6bc None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:29.578009 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927249d-4091-4442-8f01-8c1c1fe7e6bc None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:29.589374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:29.655208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fc1b5-9bbe-43cd-bf71-6f2452ea00c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:29.655934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fc1b5-9bbe-43cd-bf71-6f2452ea00c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:29.656346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fc1b5-9bbe-43cd-bf71-6f2452ea00c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:29.656879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fc1b5-9bbe-43cd-bf71-6f2452ea00c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:29.657731 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fc1b5-9bbe-43cd-bf71-6f2452ea00c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:29.661101 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fc1b5-9bbe-43cd-bf71-6f2452ea00c1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:29.693144 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ee8f8-00a8-463d-a551-fa20525a7f8d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:29.693690 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ee8f8-00a8-463d-a551-fa20525a7f8d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:29.694286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ee8f8-00a8-463d-a551-fa20525a7f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:29.704570 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1279/2584] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:29 2022] POST /identity/v3/users => generated 284 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:29.706976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fc1b5-9bbe-43cd-bf71-6f2452ea00c1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:29.719706 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fc1b5-9bbe-43cd-bf71-6f2452ea00c1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:29.725555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5991d36-298f-4de3-b6a8-31a39218d4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:29.726496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5991d36-298f-4de3-b6a8-31a39218d4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:29.726928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5991d36-298f-4de3-b6a8-31a39218d4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:29.727391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5991d36-298f-4de3-b6a8-31a39218d4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:29.728217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5991d36-298f-4de3-b6a8-31a39218d4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:29.730022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5991d36-298f-4de3-b6a8-31a39218d4b5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:29.731689 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-220ee8f8-00a8-463d-a551-fa20525a7f8d None None] MFA Rules not processed for user `3f602ff23d6141bc9d18655986c3044e`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:29.733184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-220ee8f8-00a8-463d-a551-fa20525a7f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:29.747515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1280/2585] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:29.753475 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5991d36-298f-4de3-b6a8-31a39218d4b5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:29.764880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5991d36-298f-4de3-b6a8-31a39218d4b5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:29.769800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6be6f-2ffd-4a78-99cc-55499321ed2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:29.770458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6be6f-2ffd-4a78-99cc-55499321ed2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:29.770735 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6be6f-2ffd-4a78-99cc-55499321ed2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:29.771007 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6be6f-2ffd-4a78-99cc-55499321ed2e admin admin] PATH_INFO: `/v3/projects/13c07dc53faa400d8e603e114e687a67/users/151966d2077046d08e5e0816fc5884fc/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:29.775489 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:29.804552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6be6f-2ffd-4a78-99cc-55499321ed2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c07dc53faa400d8e603e114e687a67, user_id=151966d2077046d08e5e0816fc5884fc, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:29.810999 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6be6f-2ffd-4a78-99cc-55499321ed2e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:29.834127 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1281/2586] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:29 2022] PUT /identity/v3/projects/13c07dc53faa400d8e603e114e687a67/users/151966d2077046d08e5e0816fc5884fc/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:29.839740 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6be6f-2ffd-4a78-99cc-55499321ed2e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:29.848234 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1306/2587] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:29 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:29.944546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6be6f-2ffd-4a78-99cc-55499321ed2e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:29.951452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598562a-af16-42b6-b1da-c2a2b8d50187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:29.953139 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598562a-af16-42b6-b1da-c2a2b8d50187 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:29.953590 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598562a-af16-42b6-b1da-c2a2b8d50187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:29.954059 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598562a-af16-42b6-b1da-c2a2b8d50187 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:29.955561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598562a-af16-42b6-b1da-c2a2b8d50187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:29.957827 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598562a-af16-42b6-b1da-c2a2b8d50187 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:29.969280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1282/2588] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:09:29.976143 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ee8f8-00a8-463d-a551-fa20525a7f8d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:29.987775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ee8f8-00a8-463d-a551-fa20525a7f8d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:29.993000 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0786c-f6ab-4cf0-b768-e3d02f7770ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:29.994017 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0786c-f6ab-4cf0-b768-e3d02f7770ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:29.994446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0786c-f6ab-4cf0-b768-e3d02f7770ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:29.994866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0786c-f6ab-4cf0-b768-e3d02f7770ee admin admin] PATH_INFO: `/v3/projects/13c07dc53faa400d8e603e114e687a67/users/151966d2077046d08e5e0816fc5884fc/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:29.997528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0786c-f6ab-4cf0-b768-e3d02f7770ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c07dc53faa400d8e603e114e687a67, user_id=151966d2077046d08e5e0816fc5884fc, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:30.000422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0786c-f6ab-4cf0-b768-e3d02f7770ee admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:30.019094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1307/2589] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:29 2022] PUT /identity/v3/projects/13c07dc53faa400d8e603e114e687a67/users/151966d2077046d08e5e0816fc5884fc/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:30.031758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf04754-9c8e-4e95-8457-0fcf4b66bbb7 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:30.032336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf04754-9c8e-4e95-8457-0fcf4b66bbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:30.032336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf04754-9c8e-4e95-8457-0fcf4b66bbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:30.066844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-bbf04754-9c8e-4e95-8457-0fcf4b66bbb7 None None] MFA Rules not processed for user `151966d2077046d08e5e0816fc5884fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:30.068268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bbf04754-9c8e-4e95-8457-0fcf4b66bbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:30.131079 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:30.155148 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1283/2590] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:30.167152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0786c-f6ab-4cf0-b768-e3d02f7770ee admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:30.168997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-89e0786c-f6ab-4cf0-b768-e3d02f7770ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:30.234617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0786c-f6ab-4cf0-b768-e3d02f7770ee admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:30.238882 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38794693-7099-4942-acb4-7d13b3ed66da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:30.239493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38794693-7099-4942-acb4-7d13b3ed66da service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:30.239767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38794693-7099-4942-acb4-7d13b3ed66da service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:30.240060 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38794693-7099-4942-acb4-7d13b3ed66da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:30.249697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38794693-7099-4942-acb4-7d13b3ed66da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:30.251203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38794693-7099-4942-acb4-7d13b3ed66da service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:30.262239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1308/2591] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:30.345395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf04754-9c8e-4e95-8457-0fcf4b66bbb7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:30.356801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf04754-9c8e-4e95-8457-0fcf4b66bbb7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:30.360622 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61311282-81f3-4917-993d-b6a3cccbbccc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:30.361887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61311282-81f3-4917-993d-b6a3cccbbccc service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:30.362174 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61311282-81f3-4917-993d-b6a3cccbbccc service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:30.362533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61311282-81f3-4917-993d-b6a3cccbbccc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:30.437387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61311282-81f3-4917-993d-b6a3cccbbccc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:30.439946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61311282-81f3-4917-993d-b6a3cccbbccc service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:30.451424 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1284/2592] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:30.657119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38794693-7099-4942-acb4-7d13b3ed66da service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:30.658786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-38794693-7099-4942-acb4-7d13b3ed66da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:30.758391 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38794693-7099-4942-acb4-7d13b3ed66da service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:30.764816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fff44-e626-4d89-9b6b-25e437752a66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:30.765593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fff44-e626-4d89-9b6b-25e437752a66 service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:30.765947 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fff44-e626-4d89-9b6b-25e437752a66 service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:30.766296 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fff44-e626-4d89-9b6b-25e437752a66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:30.768862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7a0fff44-e626-4d89-9b6b-25e437752a66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:30.782268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fff44-e626-4d89-9b6b-25e437752a66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:30.784028 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fff44-e626-4d89-9b6b-25e437752a66 service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:30.795721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1309/2593] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:31.282610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61311282-81f3-4917-993d-b6a3cccbbccc service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:31.357044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61311282-81f3-4917-993d-b6a3cccbbccc service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:31.361778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:31.362423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:31.362709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:31.362984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] PATH_INFO: `/v3/users/6ac5add42b8846aa87cf5e115db6fa3a` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:31.364191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ac5add42b8846aa87cf5e115db6fa3a)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:31.366159 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:31.418463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ac5add42b8846aa87cf5e115db6fa3a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:31.426453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ac5add42b8846aa87cf5e115db6fa3a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:31.441219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ac5add42b8846aa87cf5e115db6fa3a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:31.441949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ac5add42b8846aa87cf5e115db6fa3a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:31.442557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ac5add42b8846aa87cf5e115db6fa3a'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:31.456250 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1285/2594] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:31 2022] DELETE /identity/v3/users/6ac5add42b8846aa87cf5e115db6fa3a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:31.638950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fff44-e626-4d89-9b6b-25e437752a66 service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:31.652555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fff44-e626-4d89-9b6b-25e437752a66 service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:31.705965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:31.729454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:31.730558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:31.731270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:31.731858 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] PATH_INFO: `/v3/projects/3116b0dd174541268ee6d2a7605f615c` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:31.733605 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3116b0dd174541268ee6d2a7605f615c)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:31.735618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:31.796822 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3116b0dd174541268ee6d2a7605f615c', 'request_id': 'req-c6095b87-de53-4df7-8930-fdb186d5f83f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:31.805297 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3116b0dd174541268ee6d2a7605f615c', 'request_id': 'req-c6095b87-de53-4df7-8930-fdb186d5f83f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:31.814396 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3116b0dd174541268ee6d2a7605f615c', 'request_id': 'req-c6095b87-de53-4df7-8930-fdb186d5f83f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:31.837241 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] The token cache is being invalidate because project 3116b0dd174541268ee6d2a7605f615c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:31.838319 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:31.839016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:31.845133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1310/2595] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:31 2022] DELETE /identity/v3/projects/3116b0dd174541268ee6d2a7605f615c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:31.846577 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:31.848086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:31.878116 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3187b0-0640-45d1-b31e-ad9076c09a5a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:31.882535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:31.945527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e82d6c-707c-4aa5-aed3-65b479737906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:31.946218 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e82d6c-707c-4aa5-aed3-65b479737906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:31.946500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e82d6c-707c-4aa5-aed3-65b479737906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:31.946778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e82d6c-707c-4aa5-aed3-65b479737906 admin admin] PATH_INFO: `/v3/users/3f602ff23d6141bc9d18655986c3044e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:31.947934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e82d6c-707c-4aa5-aed3-65b479737906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f602ff23d6141bc9d18655986c3044e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:31.949903 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e82d6c-707c-4aa5-aed3-65b479737906 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:32.000700 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-79e82d6c-707c-4aa5-aed3-65b479737906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f602ff23d6141bc9d18655986c3044e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:32.008148 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-79e82d6c-707c-4aa5-aed3-65b479737906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f602ff23d6141bc9d18655986c3044e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:32.020377 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-79e82d6c-707c-4aa5-aed3-65b479737906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f602ff23d6141bc9d18655986c3044e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:32.021107 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-79e82d6c-707c-4aa5-aed3-65b479737906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f602ff23d6141bc9d18655986c3044e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:32.021718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-79e82d6c-707c-4aa5-aed3-65b479737906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f602ff23d6141bc9d18655986c3044e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:32.036848 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1286/2596] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:31 2022] DELETE /identity/v3/users/3f602ff23d6141bc9d18655986c3044e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:32.216382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:32.217847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:32.242603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6095b87-de53-4df7-8930-fdb186d5f83f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:32.304547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a59dc-c8ce-458c-9722-c242f2839f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:32.305202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a59dc-c8ce-458c-9722-c242f2839f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:32.305484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a59dc-c8ce-458c-9722-c242f2839f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:32.305757 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a59dc-c8ce-458c-9722-c242f2839f1b admin admin] PATH_INFO: `/v3/projects/1cc131699be546789ea565bbe314a062` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:32.306931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a59dc-c8ce-458c-9722-c242f2839f1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc131699be546789ea565bbe314a062)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:32.308934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a59dc-c8ce-458c-9722-c242f2839f1b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:32.359707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-807a59dc-c8ce-458c-9722-c242f2839f1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc131699be546789ea565bbe314a062', 'request_id': 'req-807a59dc-c8ce-458c-9722-c242f2839f1b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:32.366479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-807a59dc-c8ce-458c-9722-c242f2839f1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc131699be546789ea565bbe314a062', 'request_id': 'req-807a59dc-c8ce-458c-9722-c242f2839f1b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:32.373191 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-807a59dc-c8ce-458c-9722-c242f2839f1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc131699be546789ea565bbe314a062', 'request_id': 'req-807a59dc-c8ce-458c-9722-c242f2839f1b'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:32.387178 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-807a59dc-c8ce-458c-9722-c242f2839f1b admin admin] The token cache is being invalidate because project 1cc131699be546789ea565bbe314a062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:32.387469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-807a59dc-c8ce-458c-9722-c242f2839f1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:32.388156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-807a59dc-c8ce-458c-9722-c242f2839f1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:32.389742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1311/2597] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:32 2022] DELETE /identity/v3/projects/1cc131699be546789ea565bbe314a062 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:32.394826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30262f44-5f7b-4ee3-8b1b-a5520e422557 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:32.395306 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30262f44-5f7b-4ee3-8b1b-a5520e422557 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:32.395433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30262f44-5f7b-4ee3-8b1b-a5520e422557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:32.417113 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-30262f44-5f7b-4ee3-8b1b-a5520e422557 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:32.418352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-30262f44-5f7b-4ee3-8b1b-a5520e422557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:32.482655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1287/2598] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:32.490024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e3d79-724a-4c03-8af6-cb768b40879f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:32.490316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e3d79-724a-4c03-8af6-cb768b40879f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:32.490591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e3d79-724a-4c03-8af6-cb768b40879f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:32.498717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:32.515158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-663e3d79-724a-4c03-8af6-cb768b40879f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:32.516593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-663e3d79-724a-4c03-8af6-cb768b40879f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:32.525670 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1312/2599] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:32.530079 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30262f44-5f7b-4ee3-8b1b-a5520e422557 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:32.539955 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30262f44-5f7b-4ee3-8b1b-a5520e422557 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:32.544553 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea89538-5c0a-4b96-aca2-d26ac00132a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:32.545151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea89538-5c0a-4b96-aca2-d26ac00132a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:32.545434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea89538-5c0a-4b96-aca2-d26ac00132a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:32.545707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea89538-5c0a-4b96-aca2-d26ac00132a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:32.546352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea89538-5c0a-4b96-aca2-d26ac00132a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:09:32.546653 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea89538-5c0a-4b96-aca2-d26ac00132a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:32.548285 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea89538-5c0a-4b96-aca2-d26ac00132a6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:32.560058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1288/2600] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:09:32.565091 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e3d79-724a-4c03-8af6-cb768b40879f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:32.574238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e3d79-724a-4c03-8af6-cb768b40879f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:32.578919 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948aadb-af69-4e1f-bc6f-9b8f730ff39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:32.579511 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948aadb-af69-4e1f-bc6f-9b8f730ff39d admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:32.579855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948aadb-af69-4e1f-bc6f-9b8f730ff39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:32.580672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948aadb-af69-4e1f-bc6f-9b8f730ff39d admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:32.585460 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948aadb-af69-4e1f-bc6f-9b8f730ff39d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:32.587865 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948aadb-af69-4e1f-bc6f-9b8f730ff39d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:32.625358 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1313/2601] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:32 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:32.630205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea89538-5c0a-4b96-aca2-d26ac00132a6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:32.657298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea89538-5c0a-4b96-aca2-d26ac00132a6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:32.667050 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:32.718107 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e2a19-6b2e-491b-8f64-bf16d15e750d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:32.718722 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e2a19-6b2e-491b-8f64-bf16d15e750d admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:32.719006 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e2a19-6b2e-491b-8f64-bf16d15e750d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:32.719279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e2a19-6b2e-491b-8f64-bf16d15e750d admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:32.720077 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e2a19-6b2e-491b-8f64-bf16d15e750d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:32.722202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e2a19-6b2e-491b-8f64-bf16d15e750d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:32.744228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1289/2602] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:32 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:32.748752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948aadb-af69-4e1f-bc6f-9b8f730ff39d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:32.760204 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948aadb-af69-4e1f-bc6f-9b8f730ff39d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:32.765067 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f039f4-4337-4207-98c8-7e4d07714a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:32.765679 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f039f4-4337-4207-98c8-7e4d07714a27 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:32.765960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f039f4-4337-4207-98c8-7e4d07714a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:32.766235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f039f4-4337-4207-98c8-7e4d07714a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:32.766915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f039f4-4337-4207-98c8-7e4d07714a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:32.769253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f039f4-4337-4207-98c8-7e4d07714a27 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:32.781273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1314/2603] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:09:32.785673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e2a19-6b2e-491b-8f64-bf16d15e750d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:32.796498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e2a19-6b2e-491b-8f64-bf16d15e750d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:32.801458 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcca1b8-95b1-4589-9114-d9a446e291f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:32.802168 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcca1b8-95b1-4589-9114-d9a446e291f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:32.802523 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcca1b8-95b1-4589-9114-d9a446e291f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:32.802808 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcca1b8-95b1-4589-9114-d9a446e291f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:32.803464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcca1b8-95b1-4589-9114-d9a446e291f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:32.805124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcca1b8-95b1-4589-9114-d9a446e291f2 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:32.820899 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1290/2604] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:32.822141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f039f4-4337-4207-98c8-7e4d07714a27 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:32.833337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f039f4-4337-4207-98c8-7e4d07714a27 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:32.838186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cffc71-012e-4357-ac13-587c789de24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:32.838861 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cffc71-012e-4357-ac13-587c789de24a admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:32.839138 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cffc71-012e-4357-ac13-587c789de24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:32.839402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cffc71-012e-4357-ac13-587c789de24a admin admin] PATH_INFO: `/v3/projects/cbc92a46632240fba27bc0638357c5ad/users/1777cf6de13143428e39d127910213bc/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:32.856035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cffc71-012e-4357-ac13-587c789de24a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbc92a46632240fba27bc0638357c5ad, user_id=1777cf6de13143428e39d127910213bc, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:32.859003 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cffc71-012e-4357-ac13-587c789de24a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:32.897267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1315/2605] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:32 2022] PUT /identity/v3/projects/cbc92a46632240fba27bc0638357c5ad/users/1777cf6de13143428e39d127910213bc/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:32.902341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e299f-dff8-4add-bf27-d1c5e27152e7 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:32.902631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e299f-dff8-4add-bf27-d1c5e27152e7 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:32.902909 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e299f-dff8-4add-bf27-d1c5e27152e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:32.936331 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-ed3e299f-dff8-4add-bf27-d1c5e27152e7 None None] MFA Rules not processed for user `1777cf6de13143428e39d127910213bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:32.937586 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ed3e299f-dff8-4add-bf27-d1c5e27152e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:33.013696 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1291/2606] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:33.019342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cffc71-012e-4357-ac13-587c789de24a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:33.102020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cffc71-012e-4357-ac13-587c789de24a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:33.108765 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb81500-de01-45b2-958c-e4ad8362fed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:33.109647 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb81500-de01-45b2-958c-e4ad8362fed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:33.109898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb81500-de01-45b2-958c-e4ad8362fed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:33.110308 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb81500-de01-45b2-958c-e4ad8362fed1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:33.111662 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb81500-de01-45b2-958c-e4ad8362fed1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:33.113786 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb81500-de01-45b2-958c-e4ad8362fed1 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:33.149836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1316/2607] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:33 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:33.150911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e299f-dff8-4add-bf27-d1c5e27152e7 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:33.162470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e299f-dff8-4add-bf27-d1c5e27152e7 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:33.232822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fce88-fdde-4a74-8320-cb8699524243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:33.233615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fce88-fdde-4a74-8320-cb8699524243 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:33.233995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fce88-fdde-4a74-8320-cb8699524243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:33.234372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fce88-fdde-4a74-8320-cb8699524243 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:33.235128 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0fce88-fdde-4a74-8320-cb8699524243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:33.237849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0fce88-fdde-4a74-8320-cb8699524243 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:33.270608 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1292/2608] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:33 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:33.274777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb81500-de01-45b2-958c-e4ad8362fed1 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:33.296364 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb81500-de01-45b2-958c-e4ad8362fed1 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:33.308633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c9d66-ae2f-4b33-af7f-59d9d4580488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:33.309361 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c9d66-ae2f-4b33-af7f-59d9d4580488 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:33.309754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c9d66-ae2f-4b33-af7f-59d9d4580488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:33.310158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c9d66-ae2f-4b33-af7f-59d9d4580488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:33.311054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c9d66-ae2f-4b33-af7f-59d9d4580488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:33.312937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c9d66-ae2f-4b33-af7f-59d9d4580488 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:33.330432 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1317/2609] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:33.331532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fce88-fdde-4a74-8320-cb8699524243 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:33.353930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fce88-fdde-4a74-8320-cb8699524243 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:33.363162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb9ba4-5b15-42b3-9024-bd2659245777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:33.364362 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb9ba4-5b15-42b3-9024-bd2659245777 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:33.364509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb9ba4-5b15-42b3-9024-bd2659245777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:33.365143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb9ba4-5b15-42b3-9024-bd2659245777 admin admin] PATH_INFO: `/v3/projects/def5402657c34ebc864727fd92d998c7/users/e1b74288ce3046d682a1f742e93d210e/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:33.382364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:33.391502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb9ba4-5b15-42b3-9024-bd2659245777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=def5402657c34ebc864727fd92d998c7, user_id=e1b74288ce3046d682a1f742e93d210e, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:33.394989 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb9ba4-5b15-42b3-9024-bd2659245777 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:33.425522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1293/2610] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:33 2022] PUT /identity/v3/projects/def5402657c34ebc864727fd92d998c7/users/e1b74288ce3046d682a1f742e93d210e/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:33.427185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c9d66-ae2f-4b33-af7f-59d9d4580488 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:33.438336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c9d66-ae2f-4b33-af7f-59d9d4580488 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:33.505056 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97420235-bc37-41fa-a795-8ef90d734d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:33.505775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97420235-bc37-41fa-a795-8ef90d734d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:33.505963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97420235-bc37-41fa-a795-8ef90d734d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:33.506237 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97420235-bc37-41fa-a795-8ef90d734d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:33.506889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97420235-bc37-41fa-a795-8ef90d734d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:33.508575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97420235-bc37-41fa-a795-8ef90d734d07 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:33.519624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1318/2611] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:33.527463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb9ba4-5b15-42b3-9024-bd2659245777 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:33.541378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb9ba4-5b15-42b3-9024-bd2659245777 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:33.546524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca0970-40ae-4f37-970f-e281d1ae584c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:33.547223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca0970-40ae-4f37-970f-e281d1ae584c admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:33.547501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca0970-40ae-4f37-970f-e281d1ae584c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:33.547774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca0970-40ae-4f37-970f-e281d1ae584c admin admin] PATH_INFO: `/v3/projects/def5402657c34ebc864727fd92d998c7/users/e1b74288ce3046d682a1f742e93d210e/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:33.550507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca0970-40ae-4f37-970f-e281d1ae584c admin admin] RBAC: Authorizing `identity:create_grant(project_id=def5402657c34ebc864727fd92d998c7, user_id=e1b74288ce3046d682a1f742e93d210e, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:33.553309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca0970-40ae-4f37-970f-e281d1ae584c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:33.564542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1294/2612] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:33 2022] PUT /identity/v3/projects/def5402657c34ebc864727fd92d998c7/users/e1b74288ce3046d682a1f742e93d210e/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:33.574912 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce9f60-7902-4798-8fc7-3e6f0b491035 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:33.575203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce9f60-7902-4798-8fc7-3e6f0b491035 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:33.575481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce9f60-7902-4798-8fc7-3e6f0b491035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:33.596357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:33.606654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-24ce9f60-7902-4798-8fc7-3e6f0b491035 None None] MFA Rules not processed for user `e1b74288ce3046d682a1f742e93d210e`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:33.607883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-24ce9f60-7902-4798-8fc7-3e6f0b491035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:33.689751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1319/2613] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:33.701065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca0970-40ae-4f37-970f-e281d1ae584c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:33.708067 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bbca0970-40ae-4f37-970f-e281d1ae584c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:33.767957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca0970-40ae-4f37-970f-e281d1ae584c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:33.771685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22159b3-a88e-4f62-b803-ddf6bf6c3adf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:33.772611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22159b3-a88e-4f62-b803-ddf6bf6c3adf service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:33.772887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22159b3-a88e-4f62-b803-ddf6bf6c3adf service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:33.773154 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22159b3-a88e-4f62-b803-ddf6bf6c3adf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:33.782954 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22159b3-a88e-4f62-b803-ddf6bf6c3adf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:33.785065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22159b3-a88e-4f62-b803-ddf6bf6c3adf service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:33.795672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1295/2614] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:33.874067 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce9f60-7902-4798-8fc7-3e6f0b491035 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:33.884040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce9f60-7902-4798-8fc7-3e6f0b491035 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:33.887649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80a165-8181-4cfc-b624-500f0174024e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:33.888280 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80a165-8181-4cfc-b624-500f0174024e service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:33.888564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80a165-8181-4cfc-b624-500f0174024e service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:33.888839 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80a165-8181-4cfc-b624-500f0174024e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:33.950211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80a165-8181-4cfc-b624-500f0174024e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:33.951653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80a165-8181-4cfc-b624-500f0174024e service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:33.961639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1320/2615] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:43.790560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=36,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:43.957871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=58 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:45.192258 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22159b3-a88e-4f62-b803-ddf6bf6c3adf service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:45.264618 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22159b3-a88e-4f62-b803-ddf6bf6c3adf service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:45.269843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:45.270469 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:45.270749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:45.271021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] PATH_INFO: `/v3/users/1777cf6de13143428e39d127910213bc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:45.272418 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1777cf6de13143428e39d127910213bc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:45.274393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:45.334536 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1777cf6de13143428e39d127910213bc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:45.343197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1777cf6de13143428e39d127910213bc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:45.358005 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1777cf6de13143428e39d127910213bc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:45.359054 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1777cf6de13143428e39d127910213bc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:45.359837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1777cf6de13143428e39d127910213bc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:45.374119 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1296/2616] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:45 2022] DELETE /identity/v3/users/1777cf6de13143428e39d127910213bc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:45.378817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6923a-3530-491b-943a-134e8536e0ba None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:45.379263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6923a-3530-491b-943a-134e8536e0ba None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:45.379407 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6923a-3530-491b-943a-134e8536e0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:45.400966 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-80c6923a-3530-491b-943a-134e8536e0ba None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:45.404672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-80c6923a-3530-491b-943a-134e8536e0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:45.465635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1321/2617] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:45.476810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:45.478308 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:45.546350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16196a8-082e-459d-84e4-307927bb655d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:45.550248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5a82b-c7fa-43de-8b2a-8b3c7ad78364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:45.550870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5a82b-c7fa-43de-8b2a-8b3c7ad78364 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:45.551146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5a82b-c7fa-43de-8b2a-8b3c7ad78364 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:45.551414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5a82b-c7fa-43de-8b2a-8b3c7ad78364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:45.561757 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5a82b-c7fa-43de-8b2a-8b3c7ad78364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:45.563134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5a82b-c7fa-43de-8b2a-8b3c7ad78364 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:45.574778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1297/2618] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:45.883874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6923a-3530-491b-943a-134e8536e0ba None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:45.885297 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-80c6923a-3530-491b-943a-134e8536e0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:45.899152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6923a-3530-491b-943a-134e8536e0ba None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:45.903816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:45.904591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:45.904934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:45.905294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] PATH_INFO: `/v3/projects/cbc92a46632240fba27bc0638357c5ad` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:45.906539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbc92a46632240fba27bc0638357c5ad)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:45.913229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:45.966721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbc92a46632240fba27bc0638357c5ad', 'request_id': 'req-cc02f588-b20e-4af4-add7-c3305666eaaa'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:45.974072 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbc92a46632240fba27bc0638357c5ad', 'request_id': 'req-cc02f588-b20e-4af4-add7-c3305666eaaa'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:45.980646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbc92a46632240fba27bc0638357c5ad', 'request_id': 'req-cc02f588-b20e-4af4-add7-c3305666eaaa'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:45.995063 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] The token cache is being invalidate because project cbc92a46632240fba27bc0638357c5ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:45.995407 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:45.996574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:45.998217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1322/2619] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:45 2022] DELETE /identity/v3/projects/cbc92a46632240fba27bc0638357c5ad => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:46.002340 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5a82b-c7fa-43de-8b2a-8b3c7ad78364 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:46.012920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5a82b-c7fa-43de-8b2a-8b3c7ad78364 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:46.035134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:46.077836 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb25e4f-4c21-4a97-a407-d8adca086e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:46.078417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb25e4f-4c21-4a97-a407-d8adca086e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:46.078694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb25e4f-4c21-4a97-a407-d8adca086e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:46.078964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb25e4f-4c21-4a97-a407-d8adca086e1f admin admin] PATH_INFO: `/v3/users/e1b74288ce3046d682a1f742e93d210e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:46.080858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb25e4f-4c21-4a97-a407-d8adca086e1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b74288ce3046d682a1f742e93d210e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:46.082864 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb25e4f-4c21-4a97-a407-d8adca086e1f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:46.132575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9bb25e4f-4c21-4a97-a407-d8adca086e1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b74288ce3046d682a1f742e93d210e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:46.139746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9bb25e4f-4c21-4a97-a407-d8adca086e1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b74288ce3046d682a1f742e93d210e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:46.160109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9bb25e4f-4c21-4a97-a407-d8adca086e1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b74288ce3046d682a1f742e93d210e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:46.160979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9bb25e4f-4c21-4a97-a407-d8adca086e1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b74288ce3046d682a1f742e93d210e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:46.161956 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9bb25e4f-4c21-4a97-a407-d8adca086e1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b74288ce3046d682a1f742e93d210e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:46.176612 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1298/2620] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:46 2022] DELETE /identity/v3/users/e1b74288ce3046d682a1f742e93d210e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:46.536892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:46.538379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:46.563867 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02f588-b20e-4af4-add7-c3305666eaaa admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:46.652752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:46.653402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:46.653684 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:46.653963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637 admin admin] PATH_INFO: `/v3/projects/def5402657c34ebc864727fd92d998c7` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:46.655109 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=def5402657c34ebc864727fd92d998c7)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:46.657156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:46.694777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:46.716771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'def5402657c34ebc864727fd92d998c7', 'request_id': 'req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:46.723962 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'def5402657c34ebc864727fd92d998c7', 'request_id': 'req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:46.730940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'def5402657c34ebc864727fd92d998c7', 'request_id': 'req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:46.758327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637 admin admin] The token cache is being invalidate because project def5402657c34ebc864727fd92d998c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:46.758618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:46.759406 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-07bd1e5a-2cf3-44cb-93dc-32acaafa7637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:46.761566 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1323/2621] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:46 2022] DELETE /identity/v3/projects/def5402657c34ebc864727fd92d998c7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:46.768411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe4692-491a-4ca9-8a92-8ddf04a9d86e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:46.769230 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe4692-491a-4ca9-8a92-8ddf04a9d86e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:46.769703 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe4692-491a-4ca9-8a92-8ddf04a9d86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:46.800838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-84fe4692-491a-4ca9-8a92-8ddf04a9d86e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:46.802283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-84fe4692-491a-4ca9-8a92-8ddf04a9d86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:46.866938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1299/2622] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:46.876163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95809366-9277-46ae-b7be-c641e065a359 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:46.876722 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95809366-9277-46ae-b7be-c641e065a359 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:46.877073 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95809366-9277-46ae-b7be-c641e065a359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:46.899567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-95809366-9277-46ae-b7be-c641e065a359 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:46.900866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-95809366-9277-46ae-b7be-c641e065a359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:46.909577 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1324/2623] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:46.914108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe4692-491a-4ca9-8a92-8ddf04a9d86e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:46.924908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe4692-491a-4ca9-8a92-8ddf04a9d86e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:46.929674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6096e28-195e-4cdf-a9f9-b9a9e9857572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:46.930281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6096e28-195e-4cdf-a9f9-b9a9e9857572 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:46.930565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6096e28-195e-4cdf-a9f9-b9a9e9857572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:46.930867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6096e28-195e-4cdf-a9f9-b9a9e9857572 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:46.931513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6096e28-195e-4cdf-a9f9-b9a9e9857572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:09:46.931818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6096e28-195e-4cdf-a9f9-b9a9e9857572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:46.933469 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6096e28-195e-4cdf-a9f9-b9a9e9857572 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:46.946845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1300/2624] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:09:46.951010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95809366-9277-46ae-b7be-c641e065a359 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:46.961029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95809366-9277-46ae-b7be-c641e065a359 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:46.965818 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc1892-f084-4bc0-8e75-db56b449d8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:46.966422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc1892-f084-4bc0-8e75-db56b449d8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:46.966703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc1892-f084-4bc0-8e75-db56b449d8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:46.966981 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc1892-f084-4bc0-8e75-db56b449d8d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:46.967607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc1892-f084-4bc0-8e75-db56b449d8d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:46.969565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc1892-f084-4bc0-8e75-db56b449d8d7 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:47.002385 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1325/2625] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:46 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:47.003514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6096e28-195e-4cdf-a9f9-b9a9e9857572 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:47.014513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6096e28-195e-4cdf-a9f9-b9a9e9857572 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:47.044623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:47.080617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba138a50-20ca-48aa-b1d9-4ab7d43c72f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:47.081242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba138a50-20ca-48aa-b1d9-4ab7d43c72f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:47.081647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba138a50-20ca-48aa-b1d9-4ab7d43c72f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:47.081827 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba138a50-20ca-48aa-b1d9-4ab7d43c72f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:47.082469 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba138a50-20ca-48aa-b1d9-4ab7d43c72f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:47.084447 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba138a50-20ca-48aa-b1d9-4ab7d43c72f9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:47.112284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1301/2626] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:47 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:47.113533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc1892-f084-4bc0-8e75-db56b449d8d7 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:47.126059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc1892-f084-4bc0-8e75-db56b449d8d7 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:47.131089 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94224f50-af25-460f-a65d-6cabedf6e8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:47.132274 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94224f50-af25-460f-a65d-6cabedf6e8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:47.132837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94224f50-af25-460f-a65d-6cabedf6e8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:47.133124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94224f50-af25-460f-a65d-6cabedf6e8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:47.133844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94224f50-af25-460f-a65d-6cabedf6e8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:47.135542 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94224f50-af25-460f-a65d-6cabedf6e8e9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:47.147536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1326/2627] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:09:47.151796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba138a50-20ca-48aa-b1d9-4ab7d43c72f9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:47.162905 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba138a50-20ca-48aa-b1d9-4ab7d43c72f9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:47.170500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca0db6-5354-4ae7-afed-6c8203d13098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:47.171155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca0db6-5354-4ae7-afed-6c8203d13098 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:47.171435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca0db6-5354-4ae7-afed-6c8203d13098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:47.171707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca0db6-5354-4ae7-afed-6c8203d13098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:47.172399 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca0db6-5354-4ae7-afed-6c8203d13098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:47.174410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca0db6-5354-4ae7-afed-6c8203d13098 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:47.187863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1302/2628] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:47.192199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94224f50-af25-460f-a65d-6cabedf6e8e9 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:47.202595 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94224f50-af25-460f-a65d-6cabedf6e8e9 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:47.208878 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27158bdc-bca2-4dde-b63c-6a169661778a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:47.209648 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27158bdc-bca2-4dde-b63c-6a169661778a admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:47.210018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27158bdc-bca2-4dde-b63c-6a169661778a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:47.210367 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27158bdc-bca2-4dde-b63c-6a169661778a admin admin] PATH_INFO: `/v3/projects/186778a9e5a6498fa4d0f94f91920566/users/6e8ea486bf5c44bda02c65752ad42ace/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:47.228883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27158bdc-bca2-4dde-b63c-6a169661778a admin admin] RBAC: Authorizing `identity:create_grant(project_id=186778a9e5a6498fa4d0f94f91920566, user_id=6e8ea486bf5c44bda02c65752ad42ace, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:47.231822 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27158bdc-bca2-4dde-b63c-6a169661778a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:47.246245 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1327/2629] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:47 2022] PUT /identity/v3/projects/186778a9e5a6498fa4d0f94f91920566/users/6e8ea486bf5c44bda02c65752ad42ace/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:47.257478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ffdc3-4eef-434c-ae29-c712ca551e48 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:47.257861 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ffdc3-4eef-434c-ae29-c712ca551e48 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:47.258212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ffdc3-4eef-434c-ae29-c712ca551e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:47.294921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a48ffdc3-4eef-434c-ae29-c712ca551e48 None None] MFA Rules not processed for user `6e8ea486bf5c44bda02c65752ad42ace`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:47.296275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a48ffdc3-4eef-434c-ae29-c712ca551e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:47.377408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1303/2630] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:47.378872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27158bdc-bca2-4dde-b63c-6a169661778a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:47.446941 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27158bdc-bca2-4dde-b63c-6a169661778a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:47.452261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2aa19a-ea28-4308-94be-09775e28dfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:47.452872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2aa19a-ea28-4308-94be-09775e28dfef admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:47.453152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2aa19a-ea28-4308-94be-09775e28dfef admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:47.455890 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2aa19a-ea28-4308-94be-09775e28dfef admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:47.456630 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2aa19a-ea28-4308-94be-09775e28dfef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:47.458552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2aa19a-ea28-4308-94be-09775e28dfef admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:47.493707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1328/2631] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:47 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:47.495056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ffdc3-4eef-434c-ae29-c712ca551e48 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:47.507636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ffdc3-4eef-434c-ae29-c712ca551e48 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:47.566645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f435f4-74e3-4bc3-a5ea-532ebb09a0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:47.567249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f435f4-74e3-4bc3-a5ea-532ebb09a0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:47.567527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f435f4-74e3-4bc3-a5ea-532ebb09a0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:47.567796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f435f4-74e3-4bc3-a5ea-532ebb09a0e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:47.568445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f435f4-74e3-4bc3-a5ea-532ebb09a0e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:47.570353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f435f4-74e3-4bc3-a5ea-532ebb09a0e4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:47.591047 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1304/2632] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:47 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:47.595255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2aa19a-ea28-4308-94be-09775e28dfef admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:47.605724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2aa19a-ea28-4308-94be-09775e28dfef admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:47.610441 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67c83c-2bde-427f-bd61-c7b30bb750ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:47.611082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67c83c-2bde-427f-bd61-c7b30bb750ba admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:47.611386 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67c83c-2bde-427f-bd61-c7b30bb750ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:47.611667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67c83c-2bde-427f-bd61-c7b30bb750ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:47.612766 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67c83c-2bde-427f-bd61-c7b30bb750ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:47.614604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67c83c-2bde-427f-bd61-c7b30bb750ba admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:47.631353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1329/2633] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:47.632533 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f435f4-74e3-4bc3-a5ea-532ebb09a0e4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:47.643387 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f435f4-74e3-4bc3-a5ea-532ebb09a0e4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:47.648243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f14c-59c7-48b3-a773-722849d17278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:47.648886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f14c-59c7-48b3-a773-722849d17278 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:47.649172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f14c-59c7-48b3-a773-722849d17278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:47.649433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f14c-59c7-48b3-a773-722849d17278 admin admin] PATH_INFO: `/v3/projects/9640699d8c0f470a9ed793fc7501eb5c/users/48801c1aa29743d184ec3987fb2754b6/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:47.664194 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59f14c-59c7-48b3-a773-722849d17278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9640699d8c0f470a9ed793fc7501eb5c, user_id=48801c1aa29743d184ec3987fb2754b6, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:47.666858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59f14c-59c7-48b3-a773-722849d17278 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:47.681015 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1305/2634] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:47 2022] PUT /identity/v3/projects/9640699d8c0f470a9ed793fc7501eb5c/users/48801c1aa29743d184ec3987fb2754b6/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:47.685124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67c83c-2bde-427f-bd61-c7b30bb750ba admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:47.695960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67c83c-2bde-427f-bd61-c7b30bb750ba admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:47.716824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:47.762203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb822ab9-f174-41f1-939e-a797cd92a136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:47.762834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb822ab9-f174-41f1-939e-a797cd92a136 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:47.763114 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb822ab9-f174-41f1-939e-a797cd92a136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:47.763387 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb822ab9-f174-41f1-939e-a797cd92a136 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:47.764074 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb822ab9-f174-41f1-939e-a797cd92a136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:47.765763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb822ab9-f174-41f1-939e-a797cd92a136 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:47.778243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1330/2635] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:47.783166 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f14c-59c7-48b3-a773-722849d17278 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:47.788926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:47.800159 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f14c-59c7-48b3-a773-722849d17278 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:47.805312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578f7b0-7328-4107-90eb-fe0f9f876a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:47.806059 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578f7b0-7328-4107-90eb-fe0f9f876a9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:47.806411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578f7b0-7328-4107-90eb-fe0f9f876a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:47.807082 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578f7b0-7328-4107-90eb-fe0f9f876a9f admin admin] PATH_INFO: `/v3/projects/9640699d8c0f470a9ed793fc7501eb5c/users/48801c1aa29743d184ec3987fb2754b6/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:47.809042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578f7b0-7328-4107-90eb-fe0f9f876a9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9640699d8c0f470a9ed793fc7501eb5c, user_id=48801c1aa29743d184ec3987fb2754b6, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:47.811716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578f7b0-7328-4107-90eb-fe0f9f876a9f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:47.823627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1306/2636] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:47 2022] PUT /identity/v3/projects/9640699d8c0f470a9ed793fc7501eb5c/users/48801c1aa29743d184ec3987fb2754b6/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:47.835779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0459aa-6cac-4222-97dc-b82f78064063 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:47.836191 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0459aa-6cac-4222-97dc-b82f78064063 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:47.836674 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0459aa-6cac-4222-97dc-b82f78064063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:47.871810 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-ea0459aa-6cac-4222-97dc-b82f78064063 None None] MFA Rules not processed for user `48801c1aa29743d184ec3987fb2754b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:47.873100 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ea0459aa-6cac-4222-97dc-b82f78064063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:47.963490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1331/2637] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:47.977165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578f7b0-7328-4107-90eb-fe0f9f876a9f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:47.979297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3578f7b0-7328-4107-90eb-fe0f9f876a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:48.046905 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578f7b0-7328-4107-90eb-fe0f9f876a9f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:48.051052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364419e-3ff6-43d6-acfc-75a3f34ad025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:48.051926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364419e-3ff6-43d6-acfc-75a3f34ad025 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:48.052354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364419e-3ff6-43d6-acfc-75a3f34ad025 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:48.052709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364419e-3ff6-43d6-acfc-75a3f34ad025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:48.063221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364419e-3ff6-43d6-acfc-75a3f34ad025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:48.064966 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364419e-3ff6-43d6-acfc-75a3f34ad025 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:48.077048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1307/2638] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:47 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:48.147636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0459aa-6cac-4222-97dc-b82f78064063 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:48.149634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-ea0459aa-6cac-4222-97dc-b82f78064063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:48.233073 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0459aa-6cac-4222-97dc-b82f78064063 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:48.237856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:48.238601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:48.238986 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:48.239360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] PATH_INFO: `/v3/users/b91f35a090a6450c90ee08ef8b121f7c` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:48.240677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b91f35a090a6450c90ee08ef8b121f7c)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:48.242702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:48.303283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b91f35a090a6450c90ee08ef8b121f7c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:48.304170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b91f35a090a6450c90ee08ef8b121f7c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:48.305767 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:48.318890 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b91f35a090a6450c90ee08ef8b121f7c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:48.331304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b91f35a090a6450c90ee08ef8b121f7c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:48.351876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b91f35a090a6450c90ee08ef8b121f7c'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:48.359459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1332/2639] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:48 2022] DELETE /identity/v3/users/b91f35a090a6450c90ee08ef8b121f7c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:48.363905 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee0902-7422-4c4c-a974-fe45959db53d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:48.364240 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee0902-7422-4c4c-a974-fe45959db53d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:48.364512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee0902-7422-4c4c-a974-fe45959db53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:48.384535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4aee0902-7422-4c4c-a974-fe45959db53d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:48.385901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4aee0902-7422-4c4c-a974-fe45959db53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:48.462826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1308/2640] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:48.473083 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:48.474868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:48.548105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07312226-0935-421f-9986-685945b2ca14 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:48.552387 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e884d62-b463-4835-aaf5-c1ba43613544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:48.553000 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e884d62-b463-4835-aaf5-c1ba43613544 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:48.553279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e884d62-b463-4835-aaf5-c1ba43613544 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:48.553552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e884d62-b463-4835-aaf5-c1ba43613544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:48.569539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e884d62-b463-4835-aaf5-c1ba43613544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:48.571310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e884d62-b463-4835-aaf5-c1ba43613544 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:48.598080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1333/2641] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:48.776832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee0902-7422-4c4c-a974-fe45959db53d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:48.778636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4aee0902-7422-4c4c-a974-fe45959db53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:48.813235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee0902-7422-4c4c-a974-fe45959db53d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:48.820607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:48.821349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:48.821827 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:48.822105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] PATH_INFO: `/v3/users/6e8ea486bf5c44bda02c65752ad42ace` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:48.823485 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8ea486bf5c44bda02c65752ad42ace)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:48.826414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:48.883862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8ea486bf5c44bda02c65752ad42ace'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:48.891460 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8ea486bf5c44bda02c65752ad42ace'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:48.906334 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8ea486bf5c44bda02c65752ad42ace'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:48.907172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8ea486bf5c44bda02c65752ad42ace'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:48.908582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8ea486bf5c44bda02c65752ad42ace'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:48.939438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1309/2642] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:48 2022] DELETE /identity/v3/users/6e8ea486bf5c44bda02c65752ad42ace => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:48.941080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f987c7-14c1-41f0-8dee-b89828390b7d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:48.941398 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f987c7-14c1-41f0-8dee-b89828390b7d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:48.941682 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f987c7-14c1-41f0-8dee-b89828390b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:48.965208 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:48.967265 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:48.978224 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:49.065920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14b494-bb93-4765-93f9-cc607c80f98a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:49.071162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:49.072026 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:49.072327 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:49.072607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] PATH_INFO: `/v3/projects/8783ad64d8b34a1b9e899199ccb56ae8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:49.073910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8783ad64d8b34a1b9e899199ccb56ae8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:49.075908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:49.095741 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-f5f987c7-14c1-41f0-8dee-b89828390b7d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:49.097621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f5f987c7-14c1-41f0-8dee-b89828390b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:49.106184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1334/2643] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:09:49.117133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f987c7-14c1-41f0-8dee-b89828390b7d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:49.118866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f5f987c7-14c1-41f0-8dee-b89828390b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:49.135400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8783ad64d8b34a1b9e899199ccb56ae8', 'request_id': 'req-35339d02-5c86-4ebf-9880-98e1768d44c6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.145579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8783ad64d8b34a1b9e899199ccb56ae8', 'request_id': 'req-35339d02-5c86-4ebf-9880-98e1768d44c6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.153211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8783ad64d8b34a1b9e899199ccb56ae8', 'request_id': 'req-35339d02-5c86-4ebf-9880-98e1768d44c6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.165839 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] The token cache is being invalidate because project 8783ad64d8b34a1b9e899199ccb56ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:49.166155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.166819 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.169080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1310/2644] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:48 2022] DELETE /identity/v3/projects/8783ad64d8b34a1b9e899199ccb56ae8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:49.174265 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:49.175940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:49.184790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f987c7-14c1-41f0-8dee-b89828390b7d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:49.235195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe5526-a14f-4724-acd8-1930c4daaf10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:49.236314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe5526-a14f-4724-acd8-1930c4daaf10 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:49.236581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe5526-a14f-4724-acd8-1930c4daaf10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:49.237049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe5526-a14f-4724-acd8-1930c4daaf10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:49.238888 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1bbe5526-a14f-4724-acd8-1930c4daaf10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:49.249886 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35339d02-5c86-4ebf-9880-98e1768d44c6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:49.251209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe5526-a14f-4724-acd8-1930c4daaf10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:49.252840 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe5526-a14f-4724-acd8-1930c4daaf10 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:49.255072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:49.255744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:49.256047 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:49.256327 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] PATH_INFO: `/v3/users/151966d2077046d08e5e0816fc5884fc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:49.257467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=151966d2077046d08e5e0816fc5884fc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:49.259408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:49.263191 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1335/2645] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:49.305342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:49.318526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '151966d2077046d08e5e0816fc5884fc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.328165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '151966d2077046d08e5e0816fc5884fc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.357001 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '151966d2077046d08e5e0816fc5884fc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.357826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '151966d2077046d08e5e0816fc5884fc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.358503 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '151966d2077046d08e5e0816fc5884fc'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.375506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1311/2646] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:49 2022] DELETE /identity/v3/users/151966d2077046d08e5e0816fc5884fc => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:49.663970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe5526-a14f-4724-acd8-1930c4daaf10 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:49.665639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1bbe5526-a14f-4724-acd8-1930c4daaf10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:49.684991 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe5526-a14f-4724-acd8-1930c4daaf10 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:49.718396 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:49.768393 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:49.769052 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:49.769337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:49.769623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] PATH_INFO: `/v3/projects/186778a9e5a6498fa4d0f94f91920566` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:49.770980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=186778a9e5a6498fa4d0f94f91920566)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:49.773369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:49.794290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:49.799207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:49.823876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aac257-783d-4063-93b0-91ccadbdcc3f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:49.829189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be520255-53c8-4ef9-bfed-5c27411b759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:49.830180 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be520255-53c8-4ef9-bfed-5c27411b759f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:49.830531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be520255-53c8-4ef9-bfed-5c27411b759f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:49.830875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be520255-53c8-4ef9-bfed-5c27411b759f admin admin] PATH_INFO: `/v3/projects/13c07dc53faa400d8e603e114e687a67` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:49.832294 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be520255-53c8-4ef9-bfed-5c27411b759f admin admin] RBAC: Authorizing `identity:delete_project(project_id=13c07dc53faa400d8e603e114e687a67)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:49.834403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be520255-53c8-4ef9-bfed-5c27411b759f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:49.846078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '186778a9e5a6498fa4d0f94f91920566', 'request_id': 'req-c22223ea-0d5a-4129-ad21-d633bbbd53fb'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.854244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '186778a9e5a6498fa4d0f94f91920566', 'request_id': 'req-c22223ea-0d5a-4129-ad21-d633bbbd53fb'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.862400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '186778a9e5a6498fa4d0f94f91920566', 'request_id': 'req-c22223ea-0d5a-4129-ad21-d633bbbd53fb'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.878652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] The token cache is being invalidate because project 186778a9e5a6498fa4d0f94f91920566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:49.879210 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.879972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.881744 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1336/2647] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:49 2022] DELETE /identity/v3/projects/186778a9e5a6498fa4d0f94f91920566 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:49.885456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:49.886922 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:49.897223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be520255-53c8-4ef9-bfed-5c27411b759f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13c07dc53faa400d8e603e114e687a67', 'request_id': 'req-be520255-53c8-4ef9-bfed-5c27411b759f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.906617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be520255-53c8-4ef9-bfed-5c27411b759f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13c07dc53faa400d8e603e114e687a67', 'request_id': 'req-be520255-53c8-4ef9-bfed-5c27411b759f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.915445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be520255-53c8-4ef9-bfed-5c27411b759f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13c07dc53faa400d8e603e114e687a67', 'request_id': 'req-be520255-53c8-4ef9-bfed-5c27411b759f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.927846 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be520255-53c8-4ef9-bfed-5c27411b759f admin admin] The token cache is being invalidate because project 13c07dc53faa400d8e603e114e687a67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:49.928175 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be520255-53c8-4ef9-bfed-5c27411b759f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.928870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-be520255-53c8-4ef9-bfed-5c27411b759f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:49.930381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1312/2648] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:49 2022] DELETE /identity/v3/projects/13c07dc53faa400d8e603e114e687a67 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:49.935327 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894564ec-e746-4af3-85af-6a0aabc0ec97 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:49.935794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894564ec-e746-4af3-85af-6a0aabc0ec97 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:49.936170 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894564ec-e746-4af3-85af-6a0aabc0ec97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:49.958335 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-894564ec-e746-4af3-85af-6a0aabc0ec97 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:49.958535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22223ea-0d5a-4129-ad21-d633bbbd53fb admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:49.959676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-894564ec-e746-4af3-85af-6a0aabc0ec97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:50.030069 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1313/2649] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:50.031523 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e8491-255f-4376-852e-f0c96bad1083 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.031740 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:50.031869 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e8491-255f-4376-852e-f0c96bad1083 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.032099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e8491-255f-4376-852e-f0c96bad1083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.044402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.045548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.046186 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.046872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] PATH_INFO: `/v3/users/48801c1aa29743d184ec3987fb2754b6` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.048857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48801c1aa29743d184ec3987fb2754b6)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.051501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:50.053882 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5a4e8491-255f-4376-852e-f0c96bad1083 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:50.055086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5a4e8491-255f-4376-852e-f0c96bad1083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:50.064429 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1314/2650] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:09:50.068360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e8491-255f-4376-852e-f0c96bad1083 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:50.077529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e8491-255f-4376-852e-f0c96bad1083 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:50.082821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ad121-e311-4a1f-a5b0-9b7883ef4434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.083464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ad121-e311-4a1f-a5b0-9b7883ef4434 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.083774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ad121-e311-4a1f-a5b0-9b7883ef4434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.084041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ad121-e311-4a1f-a5b0-9b7883ef4434 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.084699 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ad121-e311-4a1f-a5b0-9b7883ef4434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:09:50.085023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ad121-e311-4a1f-a5b0-9b7883ef4434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.086684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ad121-e311-4a1f-a5b0-9b7883ef4434 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:50.101778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1315/2651] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 06:09:50.106210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ad121-e311-4a1f-a5b0-9b7883ef4434 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:50.107417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48801c1aa29743d184ec3987fb2754b6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:50.108370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48801c1aa29743d184ec3987fb2754b6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:50.116495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48801c1aa29743d184ec3987fb2754b6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:50.118241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ad121-e311-4a1f-a5b0-9b7883ef4434 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:50.123643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702eae2-98e4-4fb9-999f-29cfa6544396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.124381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702eae2-98e4-4fb9-999f-29cfa6544396 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.124737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702eae2-98e4-4fb9-999f-29cfa6544396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.125004 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702eae2-98e4-4fb9-999f-29cfa6544396 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.125641 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702eae2-98e4-4fb9-999f-29cfa6544396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.126494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48801c1aa29743d184ec3987fb2754b6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:50.127780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702eae2-98e4-4fb9-999f-29cfa6544396 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:50.141152 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48801c1aa29743d184ec3987fb2754b6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:50.147346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:50.148516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1337/2652] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:49 2022] DELETE /identity/v3/users/48801c1aa29743d184ec3987fb2754b6 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:50.167854 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1316/2653] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:50 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:50.172124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:50.174001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:50.248258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0607c-544c-4cd9-9ccd-3098bad51216 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:50.312653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59071178-bf5a-4b2d-b7f9-c59b0b48486d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.313276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59071178-bf5a-4b2d-b7f9-c59b0b48486d admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.314261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59071178-bf5a-4b2d-b7f9-c59b0b48486d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.314553 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59071178-bf5a-4b2d-b7f9-c59b0b48486d admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.315417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59071178-bf5a-4b2d-b7f9-c59b0b48486d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.317558 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59071178-bf5a-4b2d-b7f9-c59b0b48486d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:50.343201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1338/2654] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:50 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:50.347756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702eae2-98e4-4fb9-999f-29cfa6544396 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:50.359737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702eae2-98e4-4fb9-999f-29cfa6544396 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:50.365341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd11fa5-e12b-47d5-a827-0b4349a10722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.366059 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd11fa5-e12b-47d5-a827-0b4349a10722 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.366443 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd11fa5-e12b-47d5-a827-0b4349a10722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.366956 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd11fa5-e12b-47d5-a827-0b4349a10722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.367868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd11fa5-e12b-47d5-a827-0b4349a10722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.369470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd11fa5-e12b-47d5-a827-0b4349a10722 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:50.383078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1317/2655] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:50.386864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59071178-bf5a-4b2d-b7f9-c59b0b48486d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:50.398416 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59071178-bf5a-4b2d-b7f9-c59b0b48486d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:50.403158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23916a31-e05f-4ee3-95fb-50f79b2ca664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.403766 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23916a31-e05f-4ee3-95fb-50f79b2ca664 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.404084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23916a31-e05f-4ee3-95fb-50f79b2ca664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.404360 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23916a31-e05f-4ee3-95fb-50f79b2ca664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.404999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23916a31-e05f-4ee3-95fb-50f79b2ca664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.406589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23916a31-e05f-4ee3-95fb-50f79b2ca664 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:50.417181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1339/2656] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:50.421213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd11fa5-e12b-47d5-a827-0b4349a10722 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:50.432351 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd11fa5-e12b-47d5-a827-0b4349a10722 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:50.437205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7f74c-f6f7-44e1-a4b8-5aa32152fe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.437945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7f74c-f6f7-44e1-a4b8-5aa32152fe94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.438304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7f74c-f6f7-44e1-a4b8-5aa32152fe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.438637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7f74c-f6f7-44e1-a4b8-5aa32152fe94 admin admin] PATH_INFO: `/v3/projects/c88cae75766644f78e64a013e695399c/users/00738560bd2445b7ace3a3f9bd63df47/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.473343 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23916a31-e05f-4ee3-95fb-50f79b2ca664 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:50.475668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-23916a31-e05f-4ee3-95fb-50f79b2ca664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:50.489625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:50.492970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23916a31-e05f-4ee3-95fb-50f79b2ca664 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:50.499174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc211e-0b0c-4131-990e-17c2d449c186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.500165 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc211e-0b0c-4131-990e-17c2d449c186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.500663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc211e-0b0c-4131-990e-17c2d449c186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.500914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc211e-0b0c-4131-990e-17c2d449c186 admin admin] PATH_INFO: `/v3/projects/9640699d8c0f470a9ed793fc7501eb5c` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.503104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc211e-0b0c-4131-990e-17c2d449c186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9640699d8c0f470a9ed793fc7501eb5c)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.505664 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc211e-0b0c-4131-990e-17c2d449c186 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:50.579632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-28dc211e-0b0c-4131-990e-17c2d449c186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9640699d8c0f470a9ed793fc7501eb5c', 'request_id': 'req-28dc211e-0b0c-4131-990e-17c2d449c186'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:50.586434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-28dc211e-0b0c-4131-990e-17c2d449c186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9640699d8c0f470a9ed793fc7501eb5c', 'request_id': 'req-28dc211e-0b0c-4131-990e-17c2d449c186'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:50.593823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-28dc211e-0b0c-4131-990e-17c2d449c186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9640699d8c0f470a9ed793fc7501eb5c', 'request_id': 'req-28dc211e-0b0c-4131-990e-17c2d449c186'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:50.605022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7f74c-f6f7-44e1-a4b8-5aa32152fe94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c88cae75766644f78e64a013e695399c, user_id=00738560bd2445b7ace3a3f9bd63df47, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.607737 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7f74c-f6f7-44e1-a4b8-5aa32152fe94 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:50.608636 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-28dc211e-0b0c-4131-990e-17c2d449c186 admin admin] The token cache is being invalidate because project 9640699d8c0f470a9ed793fc7501eb5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:50.608927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-28dc211e-0b0c-4131-990e-17c2d449c186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:50.609579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-28dc211e-0b0c-4131-990e-17c2d449c186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:50.611883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1340/2657] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:50 2022] DELETE /identity/v3/projects/9640699d8c0f470a9ed793fc7501eb5c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:50.617265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a44116-737f-4f7b-80aa-947dc9809b02 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.617443 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a44116-737f-4f7b-80aa-947dc9809b02 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.617804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a44116-737f-4f7b-80aa-947dc9809b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.625343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1318/2658] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:50 2022] PUT /identity/v3/projects/c88cae75766644f78e64a013e695399c/users/00738560bd2445b7ace3a3f9bd63df47/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:50.636902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca2cae-2e68-42de-9e50-9f209265ee26 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.637197 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca2cae-2e68-42de-9e50-9f209265ee26 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.637469 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca2cae-2e68-42de-9e50-9f209265ee26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.639897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-77a44116-737f-4f7b-80aa-947dc9809b02 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:50.641263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-77a44116-737f-4f7b-80aa-947dc9809b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:50.667195 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-76ca2cae-2e68-42de-9e50-9f209265ee26 None None] MFA Rules not processed for user `00738560bd2445b7ace3a3f9bd63df47`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:50.668606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-76ca2cae-2e68-42de-9e50-9f209265ee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:50.712986 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1341/2659] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:50.721723 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbd27d-aa3c-4a63-ad94-41c0a25c8f1a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.722294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbd27d-aa3c-4a63-ad94-41c0a25c8f1a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.723598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbd27d-aa3c-4a63-ad94-41c0a25c8f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.750001 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-14bbd27d-aa3c-4a63-ad94-41c0a25c8f1a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:50.752544 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-14bbd27d-aa3c-4a63-ad94-41c0a25c8f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:50.756084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1319/2660] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:09:50.758870 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca2cae-2e68-42de-9e50-9f209265ee26 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:50.760466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-76ca2cae-2e68-42de-9e50-9f209265ee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:50.763584 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1342/2661] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:50.768293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbd27d-aa3c-4a63-ad94-41c0a25c8f1a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:50.778318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca2cae-2e68-42de-9e50-9f209265ee26 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:50.779081 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbd27d-aa3c-4a63-ad94-41c0a25c8f1a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:50.782861 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e5c6a-1fe6-4714-8153-d0e884b44be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.783462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e5c6a-1fe6-4714-8153-d0e884b44be4 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.783578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7362772-d6cb-49d9-9736-683706e24806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.783753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e5c6a-1fe6-4714-8153-d0e884b44be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.784020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e5c6a-1fe6-4714-8153-d0e884b44be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.784188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7362772-d6cb-49d9-9736-683706e24806 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.784445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7362772-d6cb-49d9-9736-683706e24806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.784651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e5c6a-1fe6-4714-8153-d0e884b44be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.784753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7362772-d6cb-49d9-9736-683706e24806 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.785348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7362772-d6cb-49d9-9736-683706e24806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:09:50.785650 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7362772-d6cb-49d9-9736-683706e24806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.786559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e5c6a-1fe6-4714-8153-d0e884b44be4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:50.787221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7362772-d6cb-49d9-9736-683706e24806 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:50.799775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1343/2662] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:09:50.804045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7362772-d6cb-49d9-9736-683706e24806 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:50.815720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7362772-d6cb-49d9-9736-683706e24806 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:50.821023 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed510d5-1753-4985-b3c9-f42c97ac1900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.821885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed510d5-1753-4985-b3c9-f42c97ac1900 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.822237 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed510d5-1753-4985-b3c9-f42c97ac1900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.822633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed510d5-1753-4985-b3c9-f42c97ac1900 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.823287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1320/2663] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:50 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:50.823394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed510d5-1753-4985-b3c9-f42c97ac1900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.825491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed510d5-1753-4985-b3c9-f42c97ac1900 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:50.827343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e5c6a-1fe6-4714-8153-d0e884b44be4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:50.858297 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1344/2664] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:50 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:50.862604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed510d5-1753-4985-b3c9-f42c97ac1900 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:50.902940 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e5c6a-1fe6-4714-8153-d0e884b44be4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:50.915978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:50.939738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:50.940665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed510d5-1753-4985-b3c9-f42c97ac1900 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:50.946928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b7d47-92b8-4b77-bf49-e16d484d1dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.947637 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b7d47-92b8-4b77-bf49-e16d484d1dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.947916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b7d47-92b8-4b77-bf49-e16d484d1dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.948220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b7d47-92b8-4b77-bf49-e16d484d1dc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.949136 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b7d47-92b8-4b77-bf49-e16d484d1dc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.950753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b7d47-92b8-4b77-bf49-e16d484d1dc8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:50.967722 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa020c-0c10-4631-8be3-83ded0665327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.968538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa020c-0c10-4631-8be3-83ded0665327 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.968805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa020c-0c10-4631-8be3-83ded0665327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.969127 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa020c-0c10-4631-8be3-83ded0665327 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.969986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa020c-0c10-4631-8be3-83ded0665327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.972802 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa020c-0c10-4631-8be3-83ded0665327 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:50.977661 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1345/2665] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:50 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:50.978789 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b7d47-92b8-4b77-bf49-e16d484d1dc8 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:50.991193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b7d47-92b8-4b77-bf49-e16d484d1dc8 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:50.996276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f9545-9bc7-4079-9adf-0a2a5df4d942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:50.996985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f9545-9bc7-4079-9adf-0a2a5df4d942 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:50.997332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f9545-9bc7-4079-9adf-0a2a5df4d942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:50.997696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f9545-9bc7-4079-9adf-0a2a5df4d942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:50.998406 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f9545-9bc7-4079-9adf-0a2a5df4d942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:50.999225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1321/2666] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:50 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:51.000100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa020c-0c10-4631-8be3-83ded0665327 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.000348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f9545-9bc7-4079-9adf-0a2a5df4d942 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.011452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa020c-0c10-4631-8be3-83ded0665327 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.014867 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1346/2667] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:51.015771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f9545-9bc7-4079-9adf-0a2a5df4d942 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.022433 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201066a-cfd5-45f6-884d-df01a40d236c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.023301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201066a-cfd5-45f6-884d-df01a40d236c admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.023637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201066a-cfd5-45f6-884d-df01a40d236c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.023971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201066a-cfd5-45f6-884d-df01a40d236c admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.024863 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201066a-cfd5-45f6-884d-df01a40d236c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.026601 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201066a-cfd5-45f6-884d-df01a40d236c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.026911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f9545-9bc7-4079-9adf-0a2a5df4d942 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.031938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f40c9-5aa5-4a53-8ed4-fce954e9a99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.032780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f40c9-5aa5-4a53-8ed4-fce954e9a99b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.033070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f40c9-5aa5-4a53-8ed4-fce954e9a99b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.033345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f40c9-5aa5-4a53-8ed4-fce954e9a99b admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.033985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f40c9-5aa5-4a53-8ed4-fce954e9a99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.035639 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f40c9-5aa5-4a53-8ed4-fce954e9a99b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.040403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1322/2668] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:51.046347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201066a-cfd5-45f6-884d-df01a40d236c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.049759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1347/2669] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:51.053868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f40c9-5aa5-4a53-8ed4-fce954e9a99b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.058798 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201066a-cfd5-45f6-884d-df01a40d236c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.064413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff092b42-63ea-4278-b954-dda9e6d55084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.064872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f40c9-5aa5-4a53-8ed4-fce954e9a99b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.065865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff092b42-63ea-4278-b954-dda9e6d55084 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.066153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff092b42-63ea-4278-b954-dda9e6d55084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.066637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff092b42-63ea-4278-b954-dda9e6d55084 admin admin] PATH_INFO: `/v3/projects/8316e64a7395445e8b83cb79e30e01ae/users/fa6a8d82b4db42e8aefd2af4810c2729/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.071208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe05723-78f7-453c-b9ed-3b122dd21fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.072163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe05723-78f7-453c-b9ed-3b122dd21fce admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.072669 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe05723-78f7-453c-b9ed-3b122dd21fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.073153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe05723-78f7-453c-b9ed-3b122dd21fce admin admin] PATH_INFO: `/v3/projects/848a8a650cc640fc84aff340ac8d2ea8/users/344dd02d08cc4f5c810fe341764f8f87/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.082237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff092b42-63ea-4278-b954-dda9e6d55084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8316e64a7395445e8b83cb79e30e01ae, user_id=fa6a8d82b4db42e8aefd2af4810c2729, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.085157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff092b42-63ea-4278-b954-dda9e6d55084 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.092719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe05723-78f7-453c-b9ed-3b122dd21fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=848a8a650cc640fc84aff340ac8d2ea8, user_id=344dd02d08cc4f5c810fe341764f8f87, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.097090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe05723-78f7-453c-b9ed-3b122dd21fce admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.099470 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1323/2670] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:51 2022] PUT /identity/v3/projects/8316e64a7395445e8b83cb79e30e01ae/users/fa6a8d82b4db42e8aefd2af4810c2729/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:51.103416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff092b42-63ea-4278-b954-dda9e6d55084 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.113329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1348/2671] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:51 2022] PUT /identity/v3/projects/848a8a650cc640fc84aff340ac8d2ea8/users/344dd02d08cc4f5c810fe341764f8f87/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:51.123457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f12dd-7624-47d3-9647-b283280228a4 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.123937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f12dd-7624-47d3-9647-b283280228a4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.124470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f12dd-7624-47d3-9647-b283280228a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.152970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6a0f12dd-7624-47d3-9647-b283280228a4 None None] MFA Rules not processed for user `344dd02d08cc4f5c810fe341764f8f87`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:51.154238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6a0f12dd-7624-47d3-9647-b283280228a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:51.171912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff092b42-63ea-4278-b954-dda9e6d55084 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.233771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b61cd8-a919-4984-b982-5c4d3d2b7e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.234388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b61cd8-a919-4984-b982-5c4d3d2b7e68 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.234664 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b61cd8-a919-4984-b982-5c4d3d2b7e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.234939 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b61cd8-a919-4984-b982-5c4d3d2b7e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.236393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b61cd8-a919-4984-b982-5c4d3d2b7e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.237879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b61cd8-a919-4984-b982-5c4d3d2b7e68 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.247983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1324/2672] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:09:51.249881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1349/2673] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:51.253624 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b61cd8-a919-4984-b982-5c4d3d2b7e68 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.255809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f12dd-7624-47d3-9647-b283280228a4 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.264477 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b61cd8-a919-4984-b982-5c4d3d2b7e68 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.266353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f12dd-7624-47d3-9647-b283280228a4 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.269384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e9f27-f3c4-4665-a3c4-ea3c832dcd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.270157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e9f27-f3c4-4665-a3c4-ea3c832dcd20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.270550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e9f27-f3c4-4665-a3c4-ea3c832dcd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.270832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e9f27-f3c4-4665-a3c4-ea3c832dcd20 admin admin] PATH_INFO: `/v3/projects/8316e64a7395445e8b83cb79e30e01ae/users/fa6a8d82b4db42e8aefd2af4810c2729/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.271329 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3460004-f1d7-4058-bad2-1d65108d3702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.271931 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3460004-f1d7-4058-bad2-1d65108d3702 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.272507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3460004-f1d7-4058-bad2-1d65108d3702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.272779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3460004-f1d7-4058-bad2-1d65108d3702 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.273223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e9f27-f3c4-4665-a3c4-ea3c832dcd20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8316e64a7395445e8b83cb79e30e01ae, user_id=fa6a8d82b4db42e8aefd2af4810c2729, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.273408 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3460004-f1d7-4058-bad2-1d65108d3702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.275418 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3460004-f1d7-4058-bad2-1d65108d3702 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.275878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e9f27-f3c4-4665-a3c4-ea3c832dcd20 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.288615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1325/2674] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:51 2022] PUT /identity/v3/projects/8316e64a7395445e8b83cb79e30e01ae/users/fa6a8d82b4db42e8aefd2af4810c2729/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:51.299464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbdfc2-7dd4-464f-a6f2-ce78a819855b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.299873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbdfc2-7dd4-464f-a6f2-ce78a819855b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.300175 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbdfc2-7dd4-464f-a6f2-ce78a819855b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.308742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1350/2675] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:51 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:51.312623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3460004-f1d7-4058-bad2-1d65108d3702 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.331571 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e9fbdfc2-7dd4-464f-a6f2-ce78a819855b None None] MFA Rules not processed for user `fa6a8d82b4db42e8aefd2af4810c2729`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:51.332814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e9fbdfc2-7dd4-464f-a6f2-ce78a819855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:51.342650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:51.376229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:51.381960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3460004-f1d7-4058-bad2-1d65108d3702 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.389435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da151efa-0f02-43aa-bc50-7d20fd51e3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.390217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da151efa-0f02-43aa-bc50-7d20fd51e3be admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.390516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da151efa-0f02-43aa-bc50-7d20fd51e3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.390791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da151efa-0f02-43aa-bc50-7d20fd51e3be admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.391887 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da151efa-0f02-43aa-bc50-7d20fd51e3be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.394071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da151efa-0f02-43aa-bc50-7d20fd51e3be admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.420192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1351/2676] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:51 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:51.420410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1326/2677] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:51.421589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da151efa-0f02-43aa-bc50-7d20fd51e3be admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.430350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbdfc2-7dd4-464f-a6f2-ce78a819855b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.431900 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e9fbdfc2-7dd4-464f-a6f2-ce78a819855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:51.434849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da151efa-0f02-43aa-bc50-7d20fd51e3be admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.440314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf4ecc-570b-4ba9-9e0a-81a9164540d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.440933 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf4ecc-570b-4ba9-9e0a-81a9164540d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.441213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf4ecc-570b-4ba9-9e0a-81a9164540d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.441486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf4ecc-570b-4ba9-9e0a-81a9164540d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.442170 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf4ecc-570b-4ba9-9e0a-81a9164540d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.444255 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf4ecc-570b-4ba9-9e0a-81a9164540d5 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.455850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1352/2678] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:51.459707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf4ecc-570b-4ba9-9e0a-81a9164540d5 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.469707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf4ecc-570b-4ba9-9e0a-81a9164540d5 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.475538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c4f80-fbe7-4fce-9920-98228148d307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.476231 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c4f80-fbe7-4fce-9920-98228148d307 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.476506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c4f80-fbe7-4fce-9920-98228148d307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.476775 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c4f80-fbe7-4fce-9920-98228148d307 admin admin] PATH_INFO: `/v3/projects/e0a930a665824911bd1069b222afa251/users/fcdd5749832144dfaaddd620f5a00c8d/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.493867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbdfc2-7dd4-464f-a6f2-ce78a819855b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.494659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c4f80-fbe7-4fce-9920-98228148d307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a930a665824911bd1069b222afa251, user_id=fcdd5749832144dfaaddd620f5a00c8d, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.497377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c4f80-fbe7-4fce-9920-98228148d307 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.497697 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dff5af-2a13-48ae-bba5-545e6d42dcea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.498358 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dff5af-2a13-48ae-bba5-545e6d42dcea service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.498709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dff5af-2a13-48ae-bba5-545e6d42dcea service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.499037 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dff5af-2a13-48ae-bba5-545e6d42dcea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.510719 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dff5af-2a13-48ae-bba5-545e6d42dcea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.512225 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dff5af-2a13-48ae-bba5-545e6d42dcea service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.512362 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1353/2679] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:51 2022] PUT /identity/v3/projects/e0a930a665824911bd1069b222afa251/users/fcdd5749832144dfaaddd620f5a00c8d/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:51.516302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c4f80-fbe7-4fce-9920-98228148d307 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.523333 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1327/2680] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:09:51.585491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c4f80-fbe7-4fce-9920-98228148d307 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.590676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba71c6-aac8-4061-bb57-74a5ca4f388b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.591384 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba71c6-aac8-4061-bb57-74a5ca4f388b admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.591798 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba71c6-aac8-4061-bb57-74a5ca4f388b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.592261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba71c6-aac8-4061-bb57-74a5ca4f388b admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.593053 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba71c6-aac8-4061-bb57-74a5ca4f388b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.595036 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba71c6-aac8-4061-bb57-74a5ca4f388b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.598535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dff5af-2a13-48ae-bba5-545e6d42dcea service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.606161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1354/2681] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:51.609937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dff5af-2a13-48ae-bba5-545e6d42dcea service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.610071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba71c6-aac8-4061-bb57-74a5ca4f388b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.614709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b845026-7286-45ae-888b-169ad0ec3485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.615484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b845026-7286-45ae-888b-169ad0ec3485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.615771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b845026-7286-45ae-888b-169ad0ec3485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.616109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b845026-7286-45ae-888b-169ad0ec3485 admin admin] PATH_INFO: `/v3/users/00738560bd2445b7ace3a3f9bd63df47` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.617419 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b845026-7286-45ae-888b-169ad0ec3485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00738560bd2445b7ace3a3f9bd63df47)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.619529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b845026-7286-45ae-888b-169ad0ec3485 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.620579 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba71c6-aac8-4061-bb57-74a5ca4f388b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.628090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aac878-09a8-4352-a255-fe18542746ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.630188 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aac878-09a8-4352-a255-fe18542746ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.630483 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aac878-09a8-4352-a255-fe18542746ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.630791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aac878-09a8-4352-a255-fe18542746ba admin admin] PATH_INFO: `/v3/projects/e0a930a665824911bd1069b222afa251/users/fcdd5749832144dfaaddd620f5a00c8d/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.633004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aac878-09a8-4352-a255-fe18542746ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a930a665824911bd1069b222afa251, user_id=fcdd5749832144dfaaddd620f5a00c8d, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.638533 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aac878-09a8-4352-a255-fe18542746ba admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.654476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1355/2682] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:51 2022] PUT /identity/v3/projects/e0a930a665824911bd1069b222afa251/users/fcdd5749832144dfaaddd620f5a00c8d/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:51.666864 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76067050-5dac-4ce2-aedf-a9bcc508ab1f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.667932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76067050-5dac-4ce2-aedf-a9bcc508ab1f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.668257 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76067050-5dac-4ce2-aedf-a9bcc508ab1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.682565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6b845026-7286-45ae-888b-169ad0ec3485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00738560bd2445b7ace3a3f9bd63df47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:51.691908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6b845026-7286-45ae-888b-169ad0ec3485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00738560bd2445b7ace3a3f9bd63df47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:51.700793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-76067050-5dac-4ce2-aedf-a9bcc508ab1f None None] MFA Rules not processed for user `fcdd5749832144dfaaddd620f5a00c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:51.702010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-76067050-5dac-4ce2-aedf-a9bcc508ab1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:51.707000 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6b845026-7286-45ae-888b-169ad0ec3485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00738560bd2445b7ace3a3f9bd63df47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:51.708063 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6b845026-7286-45ae-888b-169ad0ec3485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00738560bd2445b7ace3a3f9bd63df47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:51.708887 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6b845026-7286-45ae-888b-169ad0ec3485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00738560bd2445b7ace3a3f9bd63df47'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:51.725674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1328/2683] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:51 2022] DELETE /identity/v3/users/00738560bd2445b7ace3a3f9bd63df47 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:51.730244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0367e3e-c0c1-4fbf-804a-a9ba1e4635d0 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.730984 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0367e3e-c0c1-4fbf-804a-a9ba1e4635d0 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.731341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0367e3e-c0c1-4fbf-804a-a9ba1e4635d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.752812 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e0367e3e-c0c1-4fbf-804a-a9ba1e4635d0 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:51.755776 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e0367e3e-c0c1-4fbf-804a-a9ba1e4635d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:51.770235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:51.788038 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1356/2684] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:51.808031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76067050-5dac-4ce2-aedf-a9bcc508ab1f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.809811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-76067050-5dac-4ce2-aedf-a9bcc508ab1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:51.841874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:51.842330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1329/2685] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:09:51.858555 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0367e3e-c0c1-4fbf-804a-a9ba1e4635d0 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:51.860075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e0367e3e-c0c1-4fbf-804a-a9ba1e4635d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:51.889175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76067050-5dac-4ce2-aedf-a9bcc508ab1f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.919396 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0367e3e-c0c1-4fbf-804a-a9ba1e4635d0 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:51.923967 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e53bc2-b06a-485a-8ce6-8d9d2afc55ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.924689 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e53bc2-b06a-485a-8ce6-8d9d2afc55ae service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.925178 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e53bc2-b06a-485a-8ce6-8d9d2afc55ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.925393 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e53bc2-b06a-485a-8ce6-8d9d2afc55ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:51.935818 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e53bc2-b06a-485a-8ce6-8d9d2afc55ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:51.937395 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e53bc2-b06a-485a-8ce6-8d9d2afc55ae service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:51.946844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207cc87-d728-4c0a-a74f-493e76ffbb51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:51.947546 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207cc87-d728-4c0a-a74f-493e76ffbb51 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:51.947900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207cc87-d728-4c0a-a74f-493e76ffbb51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:51.948392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1330/2686] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:09:51.948509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207cc87-d728-4c0a-a74f-493e76ffbb51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:52.019319 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207cc87-d728-4c0a-a74f-493e76ffbb51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:52.020971 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207cc87-d728-4c0a-a74f-493e76ffbb51 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:52.032551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1357/2687] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:09:52.265784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e53bc2-b06a-485a-8ce6-8d9d2afc55ae service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:52.268343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f4e53bc2-b06a-485a-8ce6-8d9d2afc55ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:52.295058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e53bc2-b06a-485a-8ce6-8d9d2afc55ae service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:52.301630 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:52.302406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:52.302791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:52.303171 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] PATH_INFO: `/v3/projects/c88cae75766644f78e64a013e695399c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:52.312467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c88cae75766644f78e64a013e695399c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:52.321044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:52.375703 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c88cae75766644f78e64a013e695399c', 'request_id': 'req-b8483f7b-c190-40cb-8e2f-488173b5ef5f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.386455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c88cae75766644f78e64a013e695399c', 'request_id': 'req-b8483f7b-c190-40cb-8e2f-488173b5ef5f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.399257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c88cae75766644f78e64a013e695399c', 'request_id': 'req-b8483f7b-c190-40cb-8e2f-488173b5ef5f'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.412109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] The token cache is being invalidate because project c88cae75766644f78e64a013e695399c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:52.412460 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.413936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.418212 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1331/2688] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:52 2022] DELETE /identity/v3/projects/c88cae75766644f78e64a013e695399c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:52.420934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207cc87-d728-4c0a-a74f-493e76ffbb51 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:52.437789 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207cc87-d728-4c0a-a74f-493e76ffbb51 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:52.478727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:52.480769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:52.506883 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534870e-4902-4abf-b15c-b4497f21a5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:52.507857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534870e-4902-4abf-b15c-b4497f21a5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:52.508173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534870e-4902-4abf-b15c-b4497f21a5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:52.508442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534870e-4902-4abf-b15c-b4497f21a5d3 admin admin] PATH_INFO: `/v3/users/fa6a8d82b4db42e8aefd2af4810c2729` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:52.509583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534870e-4902-4abf-b15c-b4497f21a5d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa6a8d82b4db42e8aefd2af4810c2729)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:52.511946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534870e-4902-4abf-b15c-b4497f21a5d3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:52.539367 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:52.544220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:52.561243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8483f7b-c190-40cb-8e2f-488173b5ef5f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:52.567242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:52.567900 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:52.568210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:52.568488 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] PATH_INFO: `/v3/users/344dd02d08cc4f5c810fe341764f8f87` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:52.569646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=344dd02d08cc4f5c810fe341764f8f87)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:52.571588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:52.573275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1534870e-4902-4abf-b15c-b4497f21a5d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa6a8d82b4db42e8aefd2af4810c2729'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.574643 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1534870e-4902-4abf-b15c-b4497f21a5d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa6a8d82b4db42e8aefd2af4810c2729'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.583539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1534870e-4902-4abf-b15c-b4497f21a5d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa6a8d82b4db42e8aefd2af4810c2729'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.593972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1534870e-4902-4abf-b15c-b4497f21a5d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa6a8d82b4db42e8aefd2af4810c2729'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.616884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1534870e-4902-4abf-b15c-b4497f21a5d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa6a8d82b4db42e8aefd2af4810c2729'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.625216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1358/2689] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:52 2022] DELETE /identity/v3/users/fa6a8d82b4db42e8aefd2af4810c2729 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:52.636456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '344dd02d08cc4f5c810fe341764f8f87'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.644054 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '344dd02d08cc4f5c810fe341764f8f87'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.658650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '344dd02d08cc4f5c810fe341764f8f87'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.659493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '344dd02d08cc4f5c810fe341764f8f87'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.660516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '344dd02d08cc4f5c810fe341764f8f87'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:52.680695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1332/2690] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:52 2022] DELETE /identity/v3/users/344dd02d08cc4f5c810fe341764f8f87 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:52.682419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377e142-f264-4f4d-9962-77d91db48872 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:52.682780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377e142-f264-4f4d-9962-77d91db48872 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:52.683054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377e142-f264-4f4d-9962-77d91db48872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:52.714418 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-d377e142-f264-4f4d-9962-77d91db48872 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:52.716324 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d377e142-f264-4f4d-9962-77d91db48872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:52.796696 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1359/2691] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:52.811302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:52.816526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:52.909634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12022c0-8fd7-4384-9da8-73e55b1282e7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:52.913771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8e18c-7d72-4195-94e2-d81b21b8aeef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:52.914385 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8e18c-7d72-4195-94e2-d81b21b8aeef service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:52.914666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8e18c-7d72-4195-94e2-d81b21b8aeef service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:52.914971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8e18c-7d72-4195-94e2-d81b21b8aeef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:52.925292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8e18c-7d72-4195-94e2-d81b21b8aeef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:52.927217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8e18c-7d72-4195-94e2-d81b21b8aeef service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:52.938210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1333/2692] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:52.960410 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377e142-f264-4f4d-9962-77d91db48872 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:52.962547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d377e142-f264-4f4d-9962-77d91db48872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:52.976634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377e142-f264-4f4d-9962-77d91db48872 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:52.985476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:52.987193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:52.987585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:52.988037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48 admin admin] PATH_INFO: `/v3/projects/8316e64a7395445e8b83cb79e30e01ae` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:52.989927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8316e64a7395445e8b83cb79e30e01ae)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:52.992365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:53.061713 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8316e64a7395445e8b83cb79e30e01ae', 'request_id': 'req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.068642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8316e64a7395445e8b83cb79e30e01ae', 'request_id': 'req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.075686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8316e64a7395445e8b83cb79e30e01ae', 'request_id': 'req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.093325 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48 admin admin] The token cache is being invalidate because project 8316e64a7395445e8b83cb79e30e01ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:53.093874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.094982 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4aaac1d6-e7e4-4148-a93d-d1cd97bb1d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.101041 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1360/2693] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:52 2022] DELETE /identity/v3/projects/8316e64a7395445e8b83cb79e30e01ae => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:53.102774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693a8f6-bab4-4799-8001-77e12ad2bcea None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:53.103066 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693a8f6-bab4-4799-8001-77e12ad2bcea None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:53.103339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693a8f6-bab4-4799-8001-77e12ad2bcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:53.132927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5693a8f6-bab4-4799-8001-77e12ad2bcea None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:53.134148 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5693a8f6-bab4-4799-8001-77e12ad2bcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:53.202060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1334/2694] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:53.210141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb2bf6-2f1e-4af1-9e16-27efeec02962 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:53.210724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb2bf6-2f1e-4af1-9e16-27efeec02962 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:53.211005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb2bf6-2f1e-4af1-9e16-27efeec02962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:53.231250 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2bbb2bf6-2f1e-4af1-9e16-27efeec02962 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:53.232746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2bbb2bf6-2f1e-4af1-9e16-27efeec02962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:53.242092 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1361/2695] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:53.246649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693a8f6-bab4-4799-8001-77e12ad2bcea None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:53.255336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb2bf6-2f1e-4af1-9e16-27efeec02962 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:53.258764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693a8f6-bab4-4799-8001-77e12ad2bcea None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:53.262649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2bbb2bf6-2f1e-4af1-9e16-27efeec02962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:53.271656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ff249-7d89-4159-93d6-482a4a2aac3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:53.272710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ff249-7d89-4159-93d6-482a4a2aac3a admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:53.273546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ff249-7d89-4159-93d6-482a4a2aac3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:53.273546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ff249-7d89-4159-93d6-482a4a2aac3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:53.274444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ff249-7d89-4159-93d6-482a4a2aac3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:09:53.274843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ff249-7d89-4159-93d6-482a4a2aac3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:53.276603 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ff249-7d89-4159-93d6-482a4a2aac3a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:53.280041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:53.284109 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb2bf6-2f1e-4af1-9e16-27efeec02962 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:53.293086 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:53.294316 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1335/2696] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:09:53.294590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:53.295046 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:53.295503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] PATH_INFO: `/v3/projects/848a8a650cc640fc84aff340ac8d2ea8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:53.296698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=848a8a650cc640fc84aff340ac8d2ea8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:53.299190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ff249-7d89-4159-93d6-482a4a2aac3a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:53.307447 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:53.312574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ff249-7d89-4159-93d6-482a4a2aac3a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:53.317794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215bbc9-f3d1-4b89-89ab-7aa8741169ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:53.318510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215bbc9-f3d1-4b89-89ab-7aa8741169ce admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:53.319111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215bbc9-f3d1-4b89-89ab-7aa8741169ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:53.319491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215bbc9-f3d1-4b89-89ab-7aa8741169ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:53.320421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215bbc9-f3d1-4b89-89ab-7aa8741169ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:53.322449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215bbc9-f3d1-4b89-89ab-7aa8741169ce admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:53.325054 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:53.358019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1336/2697] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:53 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:53.363577 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215bbc9-f3d1-4b89-89ab-7aa8741169ce admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:53.366920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '848a8a650cc640fc84aff340ac8d2ea8', 'request_id': 'req-9680b7f9-1e97-446a-9650-9e9952745099'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.372727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '848a8a650cc640fc84aff340ac8d2ea8', 'request_id': 'req-9680b7f9-1e97-446a-9650-9e9952745099'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.380040 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '848a8a650cc640fc84aff340ac8d2ea8', 'request_id': 'req-9680b7f9-1e97-446a-9650-9e9952745099'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.395182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] The token cache is being invalidate because project 848a8a650cc640fc84aff340ac8d2ea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:53.395601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.396657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.398620 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1362/2698] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:53 2022] DELETE /identity/v3/projects/848a8a650cc640fc84aff340ac8d2ea8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:53.402406 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:53.403838 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:53.441001 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215bbc9-f3d1-4b89-89ab-7aa8741169ce admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:53.446644 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa50a8-4c89-4831-9c3a-7c7e83bab7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:53.447253 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa50a8-4c89-4831-9c3a-7c7e83bab7f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:53.447535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa50a8-4c89-4831-9c3a-7c7e83bab7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:53.447859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa50a8-4c89-4831-9c3a-7c7e83bab7f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:53.448527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa50a8-4c89-4831-9c3a-7c7e83bab7f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:53.450442 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa50a8-4c89-4831-9c3a-7c7e83bab7f4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:53.474597 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1337/2699] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:53 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:53.478342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa50a8-4c89-4831-9c3a-7c7e83bab7f4 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:53.480100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f7fa50a8-4c89-4831-9c3a-7c7e83bab7f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:53.481900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680b7f9-1e97-446a-9650-9e9952745099 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:53.488988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad84831-99cf-4a51-986f-cd1ec7c12945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:53.490333 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad84831-99cf-4a51-986f-cd1ec7c12945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:53.490743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad84831-99cf-4a51-986f-cd1ec7c12945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:53.490906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad84831-99cf-4a51-986f-cd1ec7c12945 admin admin] PATH_INFO: `/v3/users/fcdd5749832144dfaaddd620f5a00c8d` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:53.492104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad84831-99cf-4a51-986f-cd1ec7c12945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcdd5749832144dfaaddd620f5a00c8d)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:53.493354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa50a8-4c89-4831-9c3a-7c7e83bab7f4 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:53.494381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad84831-99cf-4a51-986f-cd1ec7c12945 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:53.498360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63f332-bb89-4560-9a1a-7cfc7342712f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:53.499232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63f332-bb89-4560-9a1a-7cfc7342712f admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:53.499889 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63f332-bb89-4560-9a1a-7cfc7342712f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:53.501990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63f332-bb89-4560-9a1a-7cfc7342712f admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:53.503279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63f332-bb89-4560-9a1a-7cfc7342712f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:53.505312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63f332-bb89-4560-9a1a-7cfc7342712f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:53.518784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1338/2700] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:09:53.523009 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63f332-bb89-4560-9a1a-7cfc7342712f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:53.534205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63f332-bb89-4560-9a1a-7cfc7342712f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:53.539236 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ec49f-c13d-491c-a1aa-2a43a9a4a047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:53.539946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ec49f-c13d-491c-a1aa-2a43a9a4a047 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:53.540350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ec49f-c13d-491c-a1aa-2a43a9a4a047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:53.540791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ec49f-c13d-491c-a1aa-2a43a9a4a047 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:53.541546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ec49f-c13d-491c-a1aa-2a43a9a4a047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:53.543211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ec49f-c13d-491c-a1aa-2a43a9a4a047 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:53.552063 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6ad84831-99cf-4a51-986f-cd1ec7c12945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcdd5749832144dfaaddd620f5a00c8d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.552711 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6ad84831-99cf-4a51-986f-cd1ec7c12945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcdd5749832144dfaaddd620f5a00c8d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.555528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1339/2701] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:09:53.559653 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ec49f-c13d-491c-a1aa-2a43a9a4a047 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:53.561412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3e8ec49f-c13d-491c-a1aa-2a43a9a4a047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:53.564494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6ad84831-99cf-4a51-986f-cd1ec7c12945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcdd5749832144dfaaddd620f5a00c8d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.574196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6ad84831-99cf-4a51-986f-cd1ec7c12945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcdd5749832144dfaaddd620f5a00c8d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.576427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ec49f-c13d-491c-a1aa-2a43a9a4a047 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:53.582210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786318a-80fe-4321-8c06-25e247e316c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:53.582937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786318a-80fe-4321-8c06-25e247e316c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:53.583219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786318a-80fe-4321-8c06-25e247e316c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:53.583522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786318a-80fe-4321-8c06-25e247e316c7 admin admin] PATH_INFO: `/v3/projects/efaf4eda83694d3b9f40edcfdcf24992/users/f78982275d4f4d55831488a52940b16c/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:53.589350 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-6ad84831-99cf-4a51-986f-cd1ec7c12945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcdd5749832144dfaaddd620f5a00c8d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:53.596624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1363/2702] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:53 2022] DELETE /identity/v3/users/fcdd5749832144dfaaddd620f5a00c8d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:53.599299 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786318a-80fe-4321-8c06-25e247e316c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efaf4eda83694d3b9f40edcfdcf24992, user_id=f78982275d4f4d55831488a52940b16c, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:53.601993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786318a-80fe-4321-8c06-25e247e316c7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:53.617182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1340/2703] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:53 2022] PUT /identity/v3/projects/efaf4eda83694d3b9f40edcfdcf24992/users/f78982275d4f4d55831488a52940b16c/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:53.630372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7140db-cac6-40c7-9047-663f3fabbbe3 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:53.630795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7140db-cac6-40c7-9047-663f3fabbbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:53.631202 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7140db-cac6-40c7-9047-663f3fabbbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:53.667018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-cb7140db-cac6-40c7-9047-663f3fabbbe3 None None] MFA Rules not processed for user `f78982275d4f4d55831488a52940b16c`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:53.668491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-cb7140db-cac6-40c7-9047-663f3fabbbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:53.679915 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:53.750697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1364/2704] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:53.756199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786318a-80fe-4321-8c06-25e247e316c7 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:53.840746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786318a-80fe-4321-8c06-25e247e316c7 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:53.846086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5567cb-5d58-436f-bdd4-175fe1ed0137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:53.846702 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5567cb-5d58-436f-bdd4-175fe1ed0137 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:53.846983 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5567cb-5d58-436f-bdd4-175fe1ed0137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:53.847253 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5567cb-5d58-436f-bdd4-175fe1ed0137 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:53.847896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5567cb-5d58-436f-bdd4-175fe1ed0137 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:53.849831 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5567cb-5d58-436f-bdd4-175fe1ed0137 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:53.882361 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1341/2705] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:53 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:53.886834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7140db-cac6-40c7-9047-663f3fabbbe3 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:53.904936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7140db-cac6-40c7-9047-663f3fabbbe3 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:53.924456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5567cb-5d58-436f-bdd4-175fe1ed0137 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:53.926602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-be5567cb-5d58-436f-bdd4-175fe1ed0137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:53.931670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:54.014678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe955af8-e578-412a-b0ee-ccaaca9f00ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.015900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe955af8-e578-412a-b0ee-ccaaca9f00ff admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.016217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe955af8-e578-412a-b0ee-ccaaca9f00ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.016495 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe955af8-e578-412a-b0ee-ccaaca9f00ff admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.017164 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe955af8-e578-412a-b0ee-ccaaca9f00ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.019088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe955af8-e578-412a-b0ee-ccaaca9f00ff admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.038838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5567cb-5d58-436f-bdd4-175fe1ed0137 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:54.045301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1365/2706] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:53 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:54.046749 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d86b5-b83e-4662-b60f-31a1902c6776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.049266 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d86b5-b83e-4662-b60f-31a1902c6776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.049782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d86b5-b83e-4662-b60f-31a1902c6776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.049914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe955af8-e578-412a-b0ee-ccaaca9f00ff admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:54.050239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d86b5-b83e-4662-b60f-31a1902c6776 admin admin] PATH_INFO: `/v3/projects/e0a930a665824911bd1069b222afa251` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.051512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d86b5-b83e-4662-b60f-31a1902c6776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0a930a665824911bd1069b222afa251)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.053558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d86b5-b83e-4662-b60f-31a1902c6776 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.059937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe955af8-e578-412a-b0ee-ccaaca9f00ff admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:54.066811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a05dc-b23f-4453-add9-6885efc319fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.067429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a05dc-b23f-4453-add9-6885efc319fa admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.067707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a05dc-b23f-4453-add9-6885efc319fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.068484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a05dc-b23f-4453-add9-6885efc319fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.069147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a05dc-b23f-4453-add9-6885efc319fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.070749 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a05dc-b23f-4453-add9-6885efc319fa admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.082944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1366/2707] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:09:54.087294 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a05dc-b23f-4453-add9-6885efc319fa admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:54.100965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a05dc-b23f-4453-add9-6885efc319fa admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:54.107312 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3f6d86b5-b83e-4662-b60f-31a1902c6776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0a930a665824911bd1069b222afa251', 'request_id': 'req-3f6d86b5-b83e-4662-b60f-31a1902c6776'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:54.117034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3f6d86b5-b83e-4662-b60f-31a1902c6776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0a930a665824911bd1069b222afa251', 'request_id': 'req-3f6d86b5-b83e-4662-b60f-31a1902c6776'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:54.123964 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3f6d86b5-b83e-4662-b60f-31a1902c6776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0a930a665824911bd1069b222afa251', 'request_id': 'req-3f6d86b5-b83e-4662-b60f-31a1902c6776'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:54.136901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3f6d86b5-b83e-4662-b60f-31a1902c6776 admin admin] The token cache is being invalidate because project e0a930a665824911bd1069b222afa251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:54.137313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3f6d86b5-b83e-4662-b60f-31a1902c6776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:54.138054 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-3f6d86b5-b83e-4662-b60f-31a1902c6776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:54.139769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1342/2708] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:53 2022] DELETE /identity/v3/projects/e0a930a665824911bd1069b222afa251 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:54.145011 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e6b0f-6aab-4485-8a75-146b6eecf285 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.145297 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e6b0f-6aab-4485-8a75-146b6eecf285 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.145877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e6b0f-6aab-4485-8a75-146b6eecf285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.167381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-af5e6b0f-6aab-4485-8a75-146b6eecf285 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:54.168620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-af5e6b0f-6aab-4485-8a75-146b6eecf285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:54.174392 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b0e41-b53f-46e1-9877-e9877557d065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.175203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b0e41-b53f-46e1-9877-e9877557d065 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.175513 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b0e41-b53f-46e1-9877-e9877557d065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.175948 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b0e41-b53f-46e1-9877-e9877557d065 admin admin] PATH_INFO: `/v3/projects/d00c1f6ecd0e4cc88a3759c05dfe6890/users/aefb6209a40848eabcf5f5c31d4b076b/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.191828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b0e41-b53f-46e1-9877-e9877557d065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d00c1f6ecd0e4cc88a3759c05dfe6890, user_id=aefb6209a40848eabcf5f5c31d4b076b, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.194928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b0e41-b53f-46e1-9877-e9877557d065 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.211160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1367/2709] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:54 2022] PUT /identity/v3/projects/d00c1f6ecd0e4cc88a3759c05dfe6890/users/aefb6209a40848eabcf5f5c31d4b076b/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:54.215016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b0e41-b53f-46e1-9877-e9877557d065 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:54.217065 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-583b0e41-b53f-46e1-9877-e9877557d065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:54.232590 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:54.237916 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1343/2710] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:09:54.245986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9ffb1-3a07-4afb-a5ad-3fc9b4c4bf09 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.246267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9ffb1-3a07-4afb-a5ad-3fc9b4c4bf09 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.246535 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9ffb1-3a07-4afb-a5ad-3fc9b4c4bf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.267767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-46a9ffb1-3a07-4afb-a5ad-3fc9b4c4bf09 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:54.269065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-46a9ffb1-3a07-4afb-a5ad-3fc9b4c4bf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:54.291955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b0e41-b53f-46e1-9877-e9877557d065 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:54.297603 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c63fb0-8b34-4345-9a0b-115a28f77636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.299472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c63fb0-8b34-4345-9a0b-115a28f77636 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.299759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c63fb0-8b34-4345-9a0b-115a28f77636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.300062 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c63fb0-8b34-4345-9a0b-115a28f77636 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.300716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c63fb0-8b34-4345-9a0b-115a28f77636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.302578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c63fb0-8b34-4345-9a0b-115a28f77636 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.302725 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:54.314816 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1368/2711] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:54.321727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c63fb0-8b34-4345-9a0b-115a28f77636 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:54.331494 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c63fb0-8b34-4345-9a0b-115a28f77636 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:54.336158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aacd15-1048-403d-a43b-4eac28bb5789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.336805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aacd15-1048-403d-a43b-4eac28bb5789 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.337082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aacd15-1048-403d-a43b-4eac28bb5789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.337353 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aacd15-1048-403d-a43b-4eac28bb5789 admin admin] PATH_INFO: `/v3/projects/d00c1f6ecd0e4cc88a3759c05dfe6890/users/aefb6209a40848eabcf5f5c31d4b076b/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.339262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aacd15-1048-403d-a43b-4eac28bb5789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d00c1f6ecd0e4cc88a3759c05dfe6890, user_id=aefb6209a40848eabcf5f5c31d4b076b, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.340388 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1344/2712] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:09:54.341973 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aacd15-1048-403d-a43b-4eac28bb5789 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.345378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9ffb1-3a07-4afb-a5ad-3fc9b4c4bf09 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:54.353875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1369/2713] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:54 2022] PUT /identity/v3/projects/d00c1f6ecd0e4cc88a3759c05dfe6890/users/aefb6209a40848eabcf5f5c31d4b076b/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:54.358169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9ffb1-3a07-4afb-a5ad-3fc9b4c4bf09 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:54.365454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a2845-ea40-4d6d-b197-bf47657e7fa6 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.365817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a2845-ea40-4d6d-b197-bf47657e7fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.366183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a2845-ea40-4d6d-b197-bf47657e7fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.403345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-1a6a2845-ea40-4d6d-b197-bf47657e7fa6 None None] MFA Rules not processed for user `aefb6209a40848eabcf5f5c31d4b076b`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:54.404871 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-1a6a2845-ea40-4d6d-b197-bf47657e7fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:54.427367 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674e36f-3d5e-4d8b-a6df-7a9022b0f131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.428105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674e36f-3d5e-4d8b-a6df-7a9022b0f131 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.428456 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674e36f-3d5e-4d8b-a6df-7a9022b0f131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.428802 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674e36f-3d5e-4d8b-a6df-7a9022b0f131 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.429987 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674e36f-3d5e-4d8b-a6df-7a9022b0f131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:09:54.430382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674e36f-3d5e-4d8b-a6df-7a9022b0f131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.432495 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674e36f-3d5e-4d8b-a6df-7a9022b0f131 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.449662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1345/2714] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 06:09:54.450457 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674e36f-3d5e-4d8b-a6df-7a9022b0f131 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:54.460740 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674e36f-3d5e-4d8b-a6df-7a9022b0f131 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:54.465772 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d032b-389b-4fe6-be5b-b7b43464254d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.466371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d032b-389b-4fe6-be5b-b7b43464254d admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.466650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d032b-389b-4fe6-be5b-b7b43464254d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.467083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d032b-389b-4fe6-be5b-b7b43464254d admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.467542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d032b-389b-4fe6-be5b-b7b43464254d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.469498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d032b-389b-4fe6-be5b-b7b43464254d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.486486 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1370/2715] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:09:54.504979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1346/2716] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:54 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:54.509260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a2845-ea40-4d6d-b197-bf47657e7fa6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:54.514412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d032b-389b-4fe6-be5b-b7b43464254d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:54.516403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b06d032b-389b-4fe6-be5b-b7b43464254d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:54.521539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a2845-ea40-4d6d-b197-bf47657e7fa6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:54.576867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d032b-389b-4fe6-be5b-b7b43464254d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:54.580268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc68078-bcf0-47b8-be83-b73685ca4b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.580879 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc68078-bcf0-47b8-be83-b73685ca4b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.581156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc68078-bcf0-47b8-be83-b73685ca4b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.581263 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302697d4-d009-4f25-92c2-c641b320dc94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.581454 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc68078-bcf0-47b8-be83-b73685ca4b53 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.581800 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302697d4-d009-4f25-92c2-c641b320dc94 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.582058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc68078-bcf0-47b8-be83-b73685ca4b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.582163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302697d4-d009-4f25-92c2-c641b320dc94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.582350 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302697d4-d009-4f25-92c2-c641b320dc94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.584026 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc68078-bcf0-47b8-be83-b73685ca4b53 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.607793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1371/2717] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:54 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:54.612926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc68078-bcf0-47b8-be83-b73685ca4b53 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:54.626138 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc68078-bcf0-47b8-be83-b73685ca4b53 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:54.632884 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a574d-9518-4fdc-8bf0-41bad4bcc845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.633514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a574d-9518-4fdc-8bf0-41bad4bcc845 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.633779 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a574d-9518-4fdc-8bf0-41bad4bcc845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.634050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a574d-9518-4fdc-8bf0-41bad4bcc845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.634734 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a574d-9518-4fdc-8bf0-41bad4bcc845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.636380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a574d-9518-4fdc-8bf0-41bad4bcc845 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.642829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:54.649590 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:54.652313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302697d4-d009-4f25-92c2-c641b320dc94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.652964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1372/2718] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:09:54.653864 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302697d4-d009-4f25-92c2-c641b320dc94 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.653985 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a574d-9518-4fdc-8bf0-41bad4bcc845 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:54.665156 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a574d-9518-4fdc-8bf0-41bad4bcc845 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:54.669979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7caf3-d928-47c4-8235-46cbea45400a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.670691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7caf3-d928-47c4-8235-46cbea45400a admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.671168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7caf3-d928-47c4-8235-46cbea45400a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.671629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7caf3-d928-47c4-8235-46cbea45400a admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.672482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7caf3-d928-47c4-8235-46cbea45400a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.674894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7caf3-d928-47c4-8235-46cbea45400a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.675271 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1347/2719] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:54.686628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1373/2720] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:54.691347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302697d4-d009-4f25-92c2-c641b320dc94 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:54.702094 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302697d4-d009-4f25-92c2-c641b320dc94 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:54.708632 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa24a54-f2c0-45bd-a688-893b3be0b67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.709304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa24a54-f2c0-45bd-a688-893b3be0b67d admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.709619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa24a54-f2c0-45bd-a688-893b3be0b67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.709840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa24a54-f2c0-45bd-a688-893b3be0b67d admin admin] PATH_INFO: `/v3/projects/c408b9e1a9864679b7fa0bc6f0f85b1d/users/59554393cd24498b8f3aae673e5099b0/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.727423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa24a54-f2c0-45bd-a688-893b3be0b67d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c408b9e1a9864679b7fa0bc6f0f85b1d, user_id=59554393cd24498b8f3aae673e5099b0, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.731076 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa24a54-f2c0-45bd-a688-893b3be0b67d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.751732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1348/2721] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:54 2022] PUT /identity/v3/projects/c408b9e1a9864679b7fa0bc6f0f85b1d/users/59554393cd24498b8f3aae673e5099b0/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:54.759578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8bd20-bbfd-4289-8e23-632edf25a84d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.759932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8bd20-bbfd-4289-8e23-632edf25a84d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.760302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8bd20-bbfd-4289-8e23-632edf25a84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.794910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-89b8bd20-bbfd-4289-8e23-632edf25a84d None None] MFA Rules not processed for user `59554393cd24498b8f3aae673e5099b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:54.796171 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-89b8bd20-bbfd-4289-8e23-632edf25a84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:54.871004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1374/2722] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:54.876852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa24a54-f2c0-45bd-a688-893b3be0b67d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:54.948814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa24a54-f2c0-45bd-a688-893b3be0b67d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:54.954498 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684347a-bf58-48b6-99c1-236b80326f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:54.955730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684347a-bf58-48b6-99c1-236b80326f29 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:54.955730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684347a-bf58-48b6-99c1-236b80326f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:54.955730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684347a-bf58-48b6-99c1-236b80326f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:54.956575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684347a-bf58-48b6-99c1-236b80326f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:54.958479 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684347a-bf58-48b6-99c1-236b80326f29 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:54.992582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1349/2723] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:54 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:54.996904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8bd20-bbfd-4289-8e23-632edf25a84d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:55.008780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8bd20-bbfd-4289-8e23-632edf25a84d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:55.063151 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684347a-bf58-48b6-99c1-236b80326f29 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:55.096830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8c1d2-7acb-4e58-8c0b-f66f30868087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:55.098047 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8c1d2-7acb-4e58-8c0b-f66f30868087 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:55.098536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8c1d2-7acb-4e58-8c0b-f66f30868087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:55.098747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8c1d2-7acb-4e58-8c0b-f66f30868087 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:55.099379 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8c1d2-7acb-4e58-8c0b-f66f30868087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:55.102478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8c1d2-7acb-4e58-8c0b-f66f30868087 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:55.128736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1375/2724] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:54 2022] POST /identity/v3/users => generated 294 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:55.132989 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8c1d2-7acb-4e58-8c0b-f66f30868087 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:55.136152 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684347a-bf58-48b6-99c1-236b80326f29 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:55.141147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd96f9-c8c5-4540-95e4-7aa61b48aea2 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefb6209a40848eabcf5f5c31d4b076b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00c1f6ecd0e4cc88a3759c05dfe6890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:55.141884 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd96f9-c8c5-4540-95e4-7aa61b48aea2 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:55.142299 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd96f9-c8c5-4540-95e4-7aa61b48aea2 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:55.142582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd96f9-c8c5-4540-95e4-7aa61b48aea2 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:55.143598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd96f9-c8c5-4540-95e4-7aa61b48aea2 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:09:55.143793 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8c1d2-7acb-4e58-8c0b-f66f30868087 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:55.143901 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd96f9-c8c5-4540-95e4-7aa61b48aea2 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:55.145478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd96f9-c8c5-4540-95e4-7aa61b48aea2 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:55.148872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517eab50-159f-4b2b-9de7-eeb9bcacab38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:55.149512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eab50-159f-4b2b-9de7-eeb9bcacab38 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:55.149795 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eab50-159f-4b2b-9de7-eeb9bcacab38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:55.150065 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eab50-159f-4b2b-9de7-eeb9bcacab38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:55.150759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517eab50-159f-4b2b-9de7-eeb9bcacab38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:55.152574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517eab50-159f-4b2b-9de7-eeb9bcacab38 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:55.159352 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1350/2725] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 06:09:55.163311 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd96f9-c8c5-4540-95e4-7aa61b48aea2 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:55.168132 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1376/2726] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:55.169071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517eab50-159f-4b2b-9de7-eeb9bcacab38 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:55.174050 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd96f9-c8c5-4540-95e4-7aa61b48aea2 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:55.179366 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517eab50-159f-4b2b-9de7-eeb9bcacab38 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:55.179659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbde49e-f516-4632-9fbb-5eb41d7071d4 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefb6209a40848eabcf5f5c31d4b076b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00c1f6ecd0e4cc88a3759c05dfe6890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:55.180285 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbde49e-f516-4632-9fbb-5eb41d7071d4 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:55.180567 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbde49e-f516-4632-9fbb-5eb41d7071d4 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:55.180841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbde49e-f516-4632-9fbb-5eb41d7071d4 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:55.181467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbde49e-f516-4632-9fbb-5eb41d7071d4 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:55.183432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbde49e-f516-4632-9fbb-5eb41d7071d4 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:55.184821 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503af1c-0e19-497d-a3f7-a8a2bd7f0e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:55.185576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503af1c-0e19-497d-a3f7-a8a2bd7f0e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:55.185956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503af1c-0e19-497d-a3f7-a8a2bd7f0e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:55.186328 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503af1c-0e19-497d-a3f7-a8a2bd7f0e81 admin admin] PATH_INFO: `/v3/projects/708f19c4b8a84e4f8e6ef1496480354e/users/21e42a1c33564624ab33b65f0b303448/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:55.203750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503af1c-0e19-497d-a3f7-a8a2bd7f0e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=708f19c4b8a84e4f8e6ef1496480354e, user_id=21e42a1c33564624ab33b65f0b303448, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:55.207817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503af1c-0e19-497d-a3f7-a8a2bd7f0e81 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:55.219109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1351/2727] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:55 2022] POST /identity/v3/projects => generated 354 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:55.220365 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbde49e-f516-4632-9fbb-5eb41d7071d4 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:55.224576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1377/2728] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:55 2022] PUT /identity/v3/projects/708f19c4b8a84e4f8e6ef1496480354e/users/21e42a1c33564624ab33b65f0b303448/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:55.228451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503af1c-0e19-497d-a3f7-a8a2bd7f0e81 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:55.254585 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:55.267688 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:55.291583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbde49e-f516-4632-9fbb-5eb41d7071d4 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:55.298435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503af1c-0e19-497d-a3f7-a8a2bd7f0e81 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:55.303514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf21329-ae6d-4b4a-91d6-e7e7bbc5c740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:55.304284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf21329-ae6d-4b4a-91d6-e7e7bbc5c740 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:55.304649 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf21329-ae6d-4b4a-91d6-e7e7bbc5c740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:55.305092 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf21329-ae6d-4b4a-91d6-e7e7bbc5c740 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:55.305754 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf21329-ae6d-4b4a-91d6-e7e7bbc5c740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:55.307449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf21329-ae6d-4b4a-91d6-e7e7bbc5c740 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:55.319549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1378/2729] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:55.323453 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf21329-ae6d-4b4a-91d6-e7e7bbc5c740 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:55.334988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf21329-ae6d-4b4a-91d6-e7e7bbc5c740 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:55.340155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5eb5b4-e398-44b9-91fc-73c9c6ceb47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:55.340946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb5b4-e398-44b9-91fc-73c9c6ceb47e admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:55.341337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb5b4-e398-44b9-91fc-73c9c6ceb47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:55.341847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb5b4-e398-44b9-91fc-73c9c6ceb47e admin admin] PATH_INFO: `/v3/projects/708f19c4b8a84e4f8e6ef1496480354e/users/21e42a1c33564624ab33b65f0b303448/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:55.346677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5eb5b4-e398-44b9-91fc-73c9c6ceb47e admin admin] RBAC: Authorizing `identity:create_grant(project_id=708f19c4b8a84e4f8e6ef1496480354e, user_id=21e42a1c33564624ab33b65f0b303448, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:55.350242 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5eb5b4-e398-44b9-91fc-73c9c6ceb47e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:55.355538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737831f-8b2f-409a-b8c4-914a8f76653f tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefb6209a40848eabcf5f5c31d4b076b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00c1f6ecd0e4cc88a3759c05dfe6890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:55.356314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737831f-8b2f-409a-b8c4-914a8f76653f tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:55.356666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737831f-8b2f-409a-b8c4-914a8f76653f tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:55.357008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737831f-8b2f-409a-b8c4-914a8f76653f tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:55.357730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737831f-8b2f-409a-b8c4-914a8f76653f tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:09:55.358152 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737831f-8b2f-409a-b8c4-914a8f76653f tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:55.360060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737831f-8b2f-409a-b8c4-914a8f76653f tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:55.362668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1379/2730] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:55 2022] PUT /identity/v3/projects/708f19c4b8a84e4f8e6ef1496480354e/users/21e42a1c33564624ab33b65f0b303448/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:55.373721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d74a39-5373-427a-bcb6-7a3ab35ef694 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:55.374002 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d74a39-5373-427a-bcb6-7a3ab35ef694 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:55.374276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d74a39-5373-427a-bcb6-7a3ab35ef694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:55.381301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1352/2731] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:09:55.408502 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-15d74a39-5373-427a-bcb6-7a3ab35ef694 None None] MFA Rules not processed for user `21e42a1c33564624ab33b65f0b303448`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:55.409812 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-15d74a39-5373-427a-bcb6-7a3ab35ef694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:55.422111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737831f-8b2f-409a-b8c4-914a8f76653f tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:55.434430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737831f-8b2f-409a-b8c4-914a8f76653f tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:55.493496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1380/2732] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:09:55.498797 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aefb6209a40848eabcf5f5c31d4b076b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00c1f6ecd0e4cc88a3759c05dfe6890', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:55.499573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:55.499859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:55.500165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] PATH_INFO: `/v3/projects/cc141d95d14c4db889760d6136d4e250` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:55.501438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cc141d95d14c4db889760d6136d4e250)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:55.503444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:55.504545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d74a39-5373-427a-bcb6-7a3ab35ef694 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:55.562170 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc141d95d14c4db889760d6136d4e250', 'request_id': 'req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:55.564485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d74a39-5373-427a-bcb6-7a3ab35ef694 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:55.581462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc141d95d14c4db889760d6136d4e250', 'request_id': 'req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:55.590493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc141d95d14c4db889760d6136d4e250', 'request_id': 'req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:55.600763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:55.606170 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] The token cache is being invalidate because project cc141d95d14c4db889760d6136d4e250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:55.606814 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:55.607955 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:55.614353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1353/2733] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:55 2022] DELETE /identity/v3/projects/cc141d95d14c4db889760d6136d4e250 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:55.621087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68826c-6c2b-43d6-85d7-cc0b8f0775d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:55.621841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68826c-6c2b-43d6-85d7-cc0b8f0775d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:55.622304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68826c-6c2b-43d6-85d7-cc0b8f0775d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:55.622648 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68826c-6c2b-43d6-85d7-cc0b8f0775d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:55.624898 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6c68826c-6c2b-43d6-85d7-cc0b8f0775d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:55.639522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:55.702029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68826c-6c2b-43d6-85d7-cc0b8f0775d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:55.703728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68826c-6c2b-43d6-85d7-cc0b8f0775d2 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:55.714886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1381/2734] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:09:55.843189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:55.845566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:55.928289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa4de0-0c28-4e21-93e6-d19256f6c8e3 tempest-VolumeQuotaClassesTest-645259938 tempest-VolumeQuotaClassesTest-645259938-project-admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:55.933520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:55.934389 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:55.934755 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:55.935118 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] PATH_INFO: `/v3/users/f78982275d4f4d55831488a52940b16c` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:55.936453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f78982275d4f4d55831488a52940b16c)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:55.939147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:56.004562 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f78982275d4f4d55831488a52940b16c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.014206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f78982275d4f4d55831488a52940b16c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.030927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f78982275d4f4d55831488a52940b16c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.031807 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f78982275d4f4d55831488a52940b16c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.032452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f78982275d4f4d55831488a52940b16c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.058176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1354/2735] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:55 2022] DELETE /identity/v3/users/f78982275d4f4d55831488a52940b16c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:56.062561 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc0703-50eb-44c2-a5f9-8112b9cbae7c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:56.063048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc0703-50eb-44c2-a5f9-8112b9cbae7c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:56.063163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc0703-50eb-44c2-a5f9-8112b9cbae7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:56.092390 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-16dc0703-50eb-44c2-a5f9-8112b9cbae7c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:56.093746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-16dc0703-50eb-44c2-a5f9-8112b9cbae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:56.171982 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1382/2736] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:56.182582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:56.184058 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:56.249540 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dea3e-890e-4ec2-ba8b-8da425c1313f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:56.254232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27a4a2-876a-42a4-bbac-c99ca377cf03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:56.254898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27a4a2-876a-42a4-bbac-c99ca377cf03 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:56.255207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27a4a2-876a-42a4-bbac-c99ca377cf03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:56.255483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27a4a2-876a-42a4-bbac-c99ca377cf03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:56.264959 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27a4a2-876a-42a4-bbac-c99ca377cf03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:56.266394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27a4a2-876a-42a4-bbac-c99ca377cf03 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:56.278565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1355/2737] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:56.562229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc0703-50eb-44c2-a5f9-8112b9cbae7c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:56.563844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-16dc0703-50eb-44c2-a5f9-8112b9cbae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:56.579372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc0703-50eb-44c2-a5f9-8112b9cbae7c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:56.584554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:56.585191 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:56.585468 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:56.585736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] PATH_INFO: `/v3/projects/efaf4eda83694d3b9f40edcfdcf24992` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:56.586855 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efaf4eda83694d3b9f40edcfdcf24992)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:56.588827 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:56.640380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efaf4eda83694d3b9f40edcfdcf24992', 'request_id': 'req-d92a01fd-d356-463e-9362-7fc010ff8c74'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.646139 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efaf4eda83694d3b9f40edcfdcf24992', 'request_id': 'req-d92a01fd-d356-463e-9362-7fc010ff8c74'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.652742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efaf4eda83694d3b9f40edcfdcf24992', 'request_id': 'req-d92a01fd-d356-463e-9362-7fc010ff8c74'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.666506 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] The token cache is being invalidate because project efaf4eda83694d3b9f40edcfdcf24992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:56.666853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.667496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.669004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1383/2738] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:56 2022] DELETE /identity/v3/projects/efaf4eda83694d3b9f40edcfdcf24992 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:56.673607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27a4a2-876a-42a4-bbac-c99ca377cf03 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:56.683349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27a4a2-876a-42a4-bbac-c99ca377cf03 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:56.701867 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:56.745692 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5639926-d67b-47b8-a87e-187ccf5d6311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:56.746464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5639926-d67b-47b8-a87e-187ccf5d6311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:56.746753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5639926-d67b-47b8-a87e-187ccf5d6311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:56.747031 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5639926-d67b-47b8-a87e-187ccf5d6311 admin admin] PATH_INFO: `/v3/users/aefb6209a40848eabcf5f5c31d4b076b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:56.748239 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5639926-d67b-47b8-a87e-187ccf5d6311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aefb6209a40848eabcf5f5c31d4b076b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:56.750164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5639926-d67b-47b8-a87e-187ccf5d6311 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:56.801314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d5639926-d67b-47b8-a87e-187ccf5d6311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aefb6209a40848eabcf5f5c31d4b076b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.808932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d5639926-d67b-47b8-a87e-187ccf5d6311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aefb6209a40848eabcf5f5c31d4b076b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.821828 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d5639926-d67b-47b8-a87e-187ccf5d6311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aefb6209a40848eabcf5f5c31d4b076b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.822608 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d5639926-d67b-47b8-a87e-187ccf5d6311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aefb6209a40848eabcf5f5c31d4b076b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.823281 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d5639926-d67b-47b8-a87e-187ccf5d6311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aefb6209a40848eabcf5f5c31d4b076b'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:56.838283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1356/2739] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:56 2022] DELETE /identity/v3/users/aefb6209a40848eabcf5f5c31d4b076b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:57.121119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:57.122704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:57.150287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a01fd-d356-463e-9362-7fc010ff8c74 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:57.205505 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:57.253894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fff8c-aa78-4595-8d40-e281d99d1c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:57.256557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fff8c-aa78-4595-8d40-e281d99d1c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:57.257118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fff8c-aa78-4595-8d40-e281d99d1c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:57.257500 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fff8c-aa78-4595-8d40-e281d99d1c60 admin admin] PATH_INFO: `/v3/projects/d00c1f6ecd0e4cc88a3759c05dfe6890` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:57.258849 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fff8c-aa78-4595-8d40-e281d99d1c60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d00c1f6ecd0e4cc88a3759c05dfe6890)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:57.261124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fff8c-aa78-4595-8d40-e281d99d1c60 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:57.326854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4b1fff8c-aa78-4595-8d40-e281d99d1c60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd00c1f6ecd0e4cc88a3759c05dfe6890', 'request_id': 'req-4b1fff8c-aa78-4595-8d40-e281d99d1c60'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:57.334818 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4b1fff8c-aa78-4595-8d40-e281d99d1c60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd00c1f6ecd0e4cc88a3759c05dfe6890', 'request_id': 'req-4b1fff8c-aa78-4595-8d40-e281d99d1c60'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:57.342312 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4b1fff8c-aa78-4595-8d40-e281d99d1c60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd00c1f6ecd0e4cc88a3759c05dfe6890', 'request_id': 'req-4b1fff8c-aa78-4595-8d40-e281d99d1c60'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:57.358573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4b1fff8c-aa78-4595-8d40-e281d99d1c60 admin admin] The token cache is being invalidate because project d00c1f6ecd0e4cc88a3759c05dfe6890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:57.358914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4b1fff8c-aa78-4595-8d40-e281d99d1c60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:57.359548 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4b1fff8c-aa78-4595-8d40-e281d99d1c60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:57.362104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1384/2740] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:57 2022] DELETE /identity/v3/projects/d00c1f6ecd0e4cc88a3759c05dfe6890 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:57.368617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e0247-b8c7-43ef-b967-beccfa0bca4e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:57.369091 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e0247-b8c7-43ef-b967-beccfa0bca4e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:57.369215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e0247-b8c7-43ef-b967-beccfa0bca4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:57.388866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f41e0247-b8c7-43ef-b967-beccfa0bca4e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:57.390089 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f41e0247-b8c7-43ef-b967-beccfa0bca4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:57.453778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1357/2741] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:57.462768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd8cf-090c-41aa-9f6b-029b783672fc None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:57.463133 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd8cf-090c-41aa-9f6b-029b783672fc None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:57.463412 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd8cf-090c-41aa-9f6b-029b783672fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:57.484013 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e4edd8cf-090c-41aa-9f6b-029b783672fc None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:57.485254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e4edd8cf-090c-41aa-9f6b-029b783672fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:57.494301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1385/2742] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:57.498779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e0247-b8c7-43ef-b967-beccfa0bca4e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:57.508898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e0247-b8c7-43ef-b967-beccfa0bca4e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:57.513514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b24eed-d285-4ea0-8d99-1da816149f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:57.514123 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24eed-d285-4ea0-8d99-1da816149f38 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:57.514403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24eed-d285-4ea0-8d99-1da816149f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:57.514676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b24eed-d285-4ea0-8d99-1da816149f38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:57.515318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b24eed-d285-4ea0-8d99-1da816149f38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:09:57.515620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b24eed-d285-4ea0-8d99-1da816149f38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:57.517325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b24eed-d285-4ea0-8d99-1da816149f38 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:57.530144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1358/2743] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:09:57.535141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edd8cf-090c-41aa-9f6b-029b783672fc None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:57.544633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edd8cf-090c-41aa-9f6b-029b783672fc None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:57.550071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7508ea-f541-4131-b92f-22f71989c94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:57.550681 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7508ea-f541-4131-b92f-22f71989c94a admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:57.550947 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7508ea-f541-4131-b92f-22f71989c94a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:57.551253 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7508ea-f541-4131-b92f-22f71989c94a admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:57.551894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7508ea-f541-4131-b92f-22f71989c94a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:57.553839 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7508ea-f541-4131-b92f-22f71989c94a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:57.583103 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1386/2744] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:57 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:57.587705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b24eed-d285-4ea0-8d99-1da816149f38 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:57.599288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b24eed-d285-4ea0-8d99-1da816149f38 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:57.628530 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:57.663515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643c10a-921f-443f-8c15-7b1399133675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:57.664168 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643c10a-921f-443f-8c15-7b1399133675 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:57.664452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643c10a-921f-443f-8c15-7b1399133675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:57.664727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643c10a-921f-443f-8c15-7b1399133675 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:57.665362 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643c10a-921f-443f-8c15-7b1399133675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:57.667292 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a643c10a-921f-443f-8c15-7b1399133675 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:57.691141 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1359/2745] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:57 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:09:57.695526 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7508ea-f541-4131-b92f-22f71989c94a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:57.707300 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7508ea-f541-4131-b92f-22f71989c94a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:57.711970 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba49b9-ce55-4e26-970c-398be4a63c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:57.712632 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba49b9-ce55-4e26-970c-398be4a63c1d admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:57.712910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba49b9-ce55-4e26-970c-398be4a63c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:57.713179 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba49b9-ce55-4e26-970c-398be4a63c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:57.713819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba49b9-ce55-4e26-970c-398be4a63c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:57.715407 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba49b9-ce55-4e26-970c-398be4a63c1d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:57.726648 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1387/2746] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:57.730954 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643c10a-921f-443f-8c15-7b1399133675 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:57.741522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643c10a-921f-443f-8c15-7b1399133675 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:57.746147 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f78715-0f8f-4177-befd-322ec3aae48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:57.746738 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f78715-0f8f-4177-befd-322ec3aae48e admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:57.747014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f78715-0f8f-4177-befd-322ec3aae48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:57.747282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f78715-0f8f-4177-befd-322ec3aae48e admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:57.748077 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f78715-0f8f-4177-befd-322ec3aae48e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:57.750157 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f78715-0f8f-4177-befd-322ec3aae48e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:57.761936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1360/2747] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:57.766297 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba49b9-ce55-4e26-970c-398be4a63c1d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:57.777309 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba49b9-ce55-4e26-970c-398be4a63c1d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:57.781953 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976892fa-edf4-4992-ab11-2b2d7ecbe514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:57.783197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976892fa-edf4-4992-ab11-2b2d7ecbe514 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:57.783470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976892fa-edf4-4992-ab11-2b2d7ecbe514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:57.783740 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976892fa-edf4-4992-ab11-2b2d7ecbe514 admin admin] PATH_INFO: `/v3/projects/259f8ccac26641cfb1d669df875b4018/users/6fd6762f3cf6494581dde7cceb7bde22/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:57.799008 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976892fa-edf4-4992-ab11-2b2d7ecbe514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=259f8ccac26641cfb1d669df875b4018, user_id=6fd6762f3cf6494581dde7cceb7bde22, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:57.801727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976892fa-edf4-4992-ab11-2b2d7ecbe514 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:57.815104 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1388/2748] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:57 2022] PUT /identity/v3/projects/259f8ccac26641cfb1d669df875b4018/users/6fd6762f3cf6494581dde7cceb7bde22/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:57.825556 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327aaf4c-3ee1-47b9-b5aa-e2197570455c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:57.825848 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327aaf4c-3ee1-47b9-b5aa-e2197570455c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:57.826149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327aaf4c-3ee1-47b9-b5aa-e2197570455c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:57.854486 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-327aaf4c-3ee1-47b9-b5aa-e2197570455c None None] MFA Rules not processed for user `6fd6762f3cf6494581dde7cceb7bde22`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:57.855698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-327aaf4c-3ee1-47b9-b5aa-e2197570455c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:57.922001 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1361/2749] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:09:57.933831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976892fa-edf4-4992-ab11-2b2d7ecbe514 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:57.935581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-976892fa-edf4-4992-ab11-2b2d7ecbe514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:57.991768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976892fa-edf4-4992-ab11-2b2d7ecbe514 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:57.995564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0e5d4-ca60-459c-90b0-57066cfa13f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:57.996259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0e5d4-ca60-459c-90b0-57066cfa13f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:57.996567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0e5d4-ca60-459c-90b0-57066cfa13f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:57.996844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0e5d4-ca60-459c-90b0-57066cfa13f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:58.005557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0e5d4-ca60-459c-90b0-57066cfa13f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:58.006986 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0e5d4-ca60-459c-90b0-57066cfa13f5 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:58.015785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1389/2750] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:58.037165 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327aaf4c-3ee1-47b9-b5aa-e2197570455c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:58.110718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327aaf4c-3ee1-47b9-b5aa-e2197570455c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:58.115293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:58.115921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:58.116244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:58.116520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] PATH_INFO: `/v3/users/6fd6762f3cf6494581dde7cceb7bde22` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:58.117652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fd6762f3cf6494581dde7cceb7bde22)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:58.119573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:58.162410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:58.169627 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fd6762f3cf6494581dde7cceb7bde22'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.177776 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fd6762f3cf6494581dde7cceb7bde22'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.192133 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fd6762f3cf6494581dde7cceb7bde22'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.193199 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fd6762f3cf6494581dde7cceb7bde22'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.193910 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fd6762f3cf6494581dde7cceb7bde22'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.208332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1362/2751] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:58 2022] DELETE /identity/v3/users/6fd6762f3cf6494581dde7cceb7bde22 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:58.213142 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56b5c1-ab2f-4ef2-923e-1049c7948408 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:58.213667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56b5c1-ab2f-4ef2-923e-1049c7948408 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:58.213773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56b5c1-ab2f-4ef2-923e-1049c7948408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:58.223089 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:58.234835 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-6d56b5c1-ab2f-4ef2-923e-1049c7948408 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:58.237233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6d56b5c1-ab2f-4ef2-923e-1049c7948408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:58.309291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1390/2752] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:58.323133 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:58.324908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:58.384427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e41ad-cc48-446f-ba57-1052ab37ed72 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:58.388517 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c40174-cb78-4303-b261-ce2f021eaa20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:58.392607 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c40174-cb78-4303-b261-ce2f021eaa20 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:58.393093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c40174-cb78-4303-b261-ce2f021eaa20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:58.393367 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c40174-cb78-4303-b261-ce2f021eaa20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:58.393614 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56b5c1-ab2f-4ef2-923e-1049c7948408 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:58.395030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-6d56b5c1-ab2f-4ef2-923e-1049c7948408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:58.403878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c40174-cb78-4303-b261-ce2f021eaa20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:58.405343 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c40174-cb78-4303-b261-ce2f021eaa20 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:58.406712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56b5c1-ab2f-4ef2-923e-1049c7948408 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:58.412080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:58.412704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:58.412983 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:58.413254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] PATH_INFO: `/v3/users/59554393cd24498b8f3aae673e5099b0` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:58.414399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=59554393cd24498b8f3aae673e5099b0)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:58.416336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:58.417307 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1363/2753] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:58.471881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59554393cd24498b8f3aae673e5099b0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.472717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59554393cd24498b8f3aae673e5099b0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.480892 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59554393cd24498b8f3aae673e5099b0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.489717 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59554393cd24498b8f3aae673e5099b0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.505790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59554393cd24498b8f3aae673e5099b0'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.512979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1391/2754] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:58 2022] DELETE /identity/v3/users/59554393cd24498b8f3aae673e5099b0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:58.517698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205784a-28b8-48b5-a065-3b3929b3fa4b None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:58.517994 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205784a-28b8-48b5-a065-3b3929b3fa4b None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:58.518272 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205784a-28b8-48b5-a065-3b3929b3fa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:58.542782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0205784a-28b8-48b5-a065-3b3929b3fa4b None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:58.544405 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0205784a-28b8-48b5-a065-3b3929b3fa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:58.620034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1364/2755] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:58.628414 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:58.630434 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:58.703452 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310449ea-af48-467b-88db-c29d6ebdbd9e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:58.708045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915a76f-3e05-4b61-bd66-43327ac10a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:58.708777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915a76f-3e05-4b61-bd66-43327ac10a95 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:58.709165 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915a76f-3e05-4b61-bd66-43327ac10a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:58.709503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915a76f-3e05-4b61-bd66-43327ac10a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:58.719380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915a76f-3e05-4b61-bd66-43327ac10a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:58.721909 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915a76f-3e05-4b61-bd66-43327ac10a95 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:58.733455 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1392/2756] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:58.765744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205784a-28b8-48b5-a065-3b3929b3fa4b None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:58.768583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0205784a-28b8-48b5-a065-3b3929b3fa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:58.795125 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205784a-28b8-48b5-a065-3b3929b3fa4b None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:58.803005 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cabf01-b94c-4277-b86a-dffdacfa7d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:58.803731 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cabf01-b94c-4277-b86a-dffdacfa7d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:58.804062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cabf01-b94c-4277-b86a-dffdacfa7d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:58.804357 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cabf01-b94c-4277-b86a-dffdacfa7d24 admin admin] PATH_INFO: `/v3/projects/259f8ccac26641cfb1d669df875b4018` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:58.805720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cabf01-b94c-4277-b86a-dffdacfa7d24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=259f8ccac26641cfb1d669df875b4018)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:58.808332 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cabf01-b94c-4277-b86a-dffdacfa7d24 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:58.873329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d9cabf01-b94c-4277-b86a-dffdacfa7d24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '259f8ccac26641cfb1d669df875b4018', 'request_id': 'req-d9cabf01-b94c-4277-b86a-dffdacfa7d24'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.878829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:58.885361 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d9cabf01-b94c-4277-b86a-dffdacfa7d24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '259f8ccac26641cfb1d669df875b4018', 'request_id': 'req-d9cabf01-b94c-4277-b86a-dffdacfa7d24'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.895052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d9cabf01-b94c-4277-b86a-dffdacfa7d24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '259f8ccac26641cfb1d669df875b4018', 'request_id': 'req-d9cabf01-b94c-4277-b86a-dffdacfa7d24'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.908976 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d9cabf01-b94c-4277-b86a-dffdacfa7d24 admin admin] The token cache is being invalidate because project 259f8ccac26641cfb1d669df875b4018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:58.909275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d9cabf01-b94c-4277-b86a-dffdacfa7d24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.909951 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-d9cabf01-b94c-4277-b86a-dffdacfa7d24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:58.912060 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1365/2757] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:58 2022] DELETE /identity/v3/projects/259f8ccac26641cfb1d669df875b4018 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:58.916638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003b718-bb8b-4814-b1cf-f76f490e635c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:58.916999 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003b718-bb8b-4814-b1cf-f76f490e635c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:58.917184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003b718-bb8b-4814-b1cf-f76f490e635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:58.938094 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b003b718-bb8b-4814-b1cf-f76f490e635c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:58.940369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b003b718-bb8b-4814-b1cf-f76f490e635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:58.967262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:59.027805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1393/2758] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:59.035639 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fdf27-a658-47ed-8e96-ed202d9f0366 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:59.035925 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fdf27-a658-47ed-8e96-ed202d9f0366 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:59.036329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fdf27-a658-47ed-8e96-ed202d9f0366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:59.058796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-395fdf27-a658-47ed-8e96-ed202d9f0366 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:59.060124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-395fdf27-a658-47ed-8e96-ed202d9f0366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:59.070050 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1366/2759] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:59.074009 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fdf27-a658-47ed-8e96-ed202d9f0366 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:59.074654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003b718-bb8b-4814-b1cf-f76f490e635c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:59.076503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b003b718-bb8b-4814-b1cf-f76f490e635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:59.085767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fdf27-a658-47ed-8e96-ed202d9f0366 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:59.090532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac061cc2-ba6e-4109-8132-244b9608b450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:59.091124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061cc2-ba6e-4109-8132-244b9608b450 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:59.091418 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061cc2-ba6e-4109-8132-244b9608b450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:59.091715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac061cc2-ba6e-4109-8132-244b9608b450 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:59.092370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac061cc2-ba6e-4109-8132-244b9608b450 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:09:59.092722 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac061cc2-ba6e-4109-8132-244b9608b450 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:59.094345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac061cc2-ba6e-4109-8132-244b9608b450 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:59.101846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003b718-bb8b-4814-b1cf-f76f490e635c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:59.111500 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1367/2760] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:09:59.112780 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac061cc2-ba6e-4109-8132-244b9608b450 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:59.116044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:59.116695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:59.116972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:59.117237 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] PATH_INFO: `/v3/projects/c408b9e1a9864679b7fa0bc6f0f85b1d` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:59.118789 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=c408b9e1a9864679b7fa0bc6f0f85b1d)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:59.121987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:59.124024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac061cc2-ba6e-4109-8132-244b9608b450 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:59.129409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f90ecc-fabc-4d9f-b451-ad0f9928007e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:59.130113 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f90ecc-fabc-4d9f-b451-ad0f9928007e admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:59.130587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f90ecc-fabc-4d9f-b451-ad0f9928007e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:59.130961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f90ecc-fabc-4d9f-b451-ad0f9928007e admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:59.131690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f90ecc-fabc-4d9f-b451-ad0f9928007e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:59.133721 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f90ecc-fabc-4d9f-b451-ad0f9928007e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:59.171152 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1368/2761] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:09:59 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:59.173768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f90ecc-fabc-4d9f-b451-ad0f9928007e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:59.184176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c408b9e1a9864679b7fa0bc6f0f85b1d', 'request_id': 'req-4ed443b7-9048-463b-b004-93d8c15ce8ea'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.193345 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c408b9e1a9864679b7fa0bc6f0f85b1d', 'request_id': 'req-4ed443b7-9048-463b-b004-93d8c15ce8ea'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.201614 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c408b9e1a9864679b7fa0bc6f0f85b1d', 'request_id': 'req-4ed443b7-9048-463b-b004-93d8c15ce8ea'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.222918 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] The token cache is being invalidate because project c408b9e1a9864679b7fa0bc6f0f85b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:59.222918 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.224227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.226002 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1394/2762] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:59 2022] DELETE /identity/v3/projects/c408b9e1a9864679b7fa0bc6f0f85b1d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:59.229709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:59.231070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:59.250924 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f90ecc-fabc-4d9f-b451-ad0f9928007e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:59.256591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a782b-7326-47ce-9431-c54e7de732a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:59.257217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a782b-7326-47ce-9431-c54e7de732a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:59.257497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a782b-7326-47ce-9431-c54e7de732a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:59.257771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a782b-7326-47ce-9431-c54e7de732a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:59.258424 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a782b-7326-47ce-9431-c54e7de732a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:59.260449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a782b-7326-47ce-9431-c54e7de732a5 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:59.287093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1369/2763] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:09:59 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:09:59.291038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a782b-7326-47ce-9431-c54e7de732a5 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:59.293133 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a58a782b-7326-47ce-9431-c54e7de732a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:59.307591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a782b-7326-47ce-9431-c54e7de732a5 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:59.315727 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db00e8-1f02-4bda-9545-f72a611cfe3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:59.316452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db00e8-1f02-4bda-9545-f72a611cfe3f admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:59.316747 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db00e8-1f02-4bda-9545-f72a611cfe3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:59.317010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db00e8-1f02-4bda-9545-f72a611cfe3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:59.317658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db00e8-1f02-4bda-9545-f72a611cfe3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:59.318182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed443b7-9048-463b-b004-93d8c15ce8ea admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:59.319342 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db00e8-1f02-4bda-9545-f72a611cfe3f admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:59.323580 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:59.324271 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:59.324716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:59.325000 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] PATH_INFO: `/v3/users/21e42a1c33564624ab33b65f0b303448` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:59.326301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e42a1c33564624ab33b65f0b303448)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:59.328341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:59.332205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1370/2764] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:09:59.335985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db00e8-1f02-4bda-9545-f72a611cfe3f admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:59.351059 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db00e8-1f02-4bda-9545-f72a611cfe3f admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:59.355861 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eeefbc-8f58-4757-b60f-9e4392d4b4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:59.356509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeefbc-8f58-4757-b60f-9e4392d4b4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:59.356796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeefbc-8f58-4757-b60f-9e4392d4b4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:59.357073 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeefbc-8f58-4757-b60f-9e4392d4b4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:59.357716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eeefbc-8f58-4757-b60f-9e4392d4b4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:59.359324 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eeefbc-8f58-4757-b60f-9e4392d4b4e3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:59.375946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1371/2765] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:09:59.386210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eeefbc-8f58-4757-b60f-9e4392d4b4e3 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:59.398452 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eeefbc-8f58-4757-b60f-9e4392d4b4e3 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:59.403265 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:59.403838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81085c6b-15e6-4862-8826-1f70e0a24558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:59.404531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085c6b-15e6-4862-8826-1f70e0a24558 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:59.404790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085c6b-15e6-4862-8826-1f70e0a24558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:59.405087 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085c6b-15e6-4862-8826-1f70e0a24558 admin admin] PATH_INFO: `/v3/projects/54eb11eb37d74f6dbd22409bdca6f82b/users/51ca25c276d24b128233d102a4c87240/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:59.405946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e42a1c33564624ab33b65f0b303448'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.406990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e42a1c33564624ab33b65f0b303448'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.418720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e42a1c33564624ab33b65f0b303448'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.420958 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81085c6b-15e6-4862-8826-1f70e0a24558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54eb11eb37d74f6dbd22409bdca6f82b, user_id=51ca25c276d24b128233d102a4c87240, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:59.424821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81085c6b-15e6-4862-8826-1f70e0a24558 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:59.427436 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e42a1c33564624ab33b65f0b303448'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.439493 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1372/2766] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:09:59 2022] PUT /identity/v3/projects/54eb11eb37d74f6dbd22409bdca6f82b/users/51ca25c276d24b128233d102a4c87240/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:59.443635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e42a1c33564624ab33b65f0b303448'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.450471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1395/2767] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:09:59 2022] DELETE /identity/v3/users/21e42a1c33564624ab33b65f0b303448 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:09:59.453465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e2254-312f-425e-91a6-3f3851b7c5ad None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:59.453465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e2254-312f-425e-91a6-3f3851b7c5ad None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:59.453465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e2254-312f-425e-91a6-3f3851b7c5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:59.484499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-539e2254-312f-425e-91a6-3f3851b7c5ad None None] MFA Rules not processed for user `51ca25c276d24b128233d102a4c87240`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:09:59.485835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-539e2254-312f-425e-91a6-3f3851b7c5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:59.526341 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:09:59.567105 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1373/2768] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:09:59.577724 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:59.579208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:59.649402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160285c-023b-4569-b397-b800a5b5ba35 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:59.653226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164efb43-e20a-4e29-805a-9efa825d75ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:59.653853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164efb43-e20a-4e29-805a-9efa825d75ed service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:59.654136 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164efb43-e20a-4e29-805a-9efa825d75ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:59.654412 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164efb43-e20a-4e29-805a-9efa825d75ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:59.664256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164efb43-e20a-4e29-805a-9efa825d75ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:59.665560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164efb43-e20a-4e29-805a-9efa825d75ed service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:59.678617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1396/2769] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:09:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:09:59.757068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e2254-312f-425e-91a6-3f3851b7c5ad None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:09:59.758740 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-539e2254-312f-425e-91a6-3f3851b7c5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:09:59.853102 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e2254-312f-425e-91a6-3f3851b7c5ad None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:09:59.858977 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:09:59.859604 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:59.859889 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:59.860271 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7 admin admin] PATH_INFO: `/v3/projects/708f19c4b8a84e4f8e6ef1496480354e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:09:59.861462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=708f19c4b8a84e4f8e6ef1496480354e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:09:59.863396 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:09:59.921546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '708f19c4b8a84e4f8e6ef1496480354e', 'request_id': 'req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.933334 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '708f19c4b8a84e4f8e6ef1496480354e', 'request_id': 'req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.941705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '708f19c4b8a84e4f8e6ef1496480354e', 'request_id': 'req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.966202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7 admin admin] The token cache is being invalidate because project 708f19c4b8a84e4f8e6ef1496480354e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:09:59.966891 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.967560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-290d526d-50ae-4a5f-9ef3-22fbf54a16e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:09:59.972542 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1374/2770] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:09:59 2022] DELETE /identity/v3/projects/708f19c4b8a84e4f8e6ef1496480354e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:09:59.974275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0531d-607b-45f1-9ab5-edbe90d93329 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:09:59.974735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0531d-607b-45f1-9ab5-edbe90d93329 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:09:59.975015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0531d-607b-45f1-9ab5-edbe90d93329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:00.002962 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-9fc0531d-607b-45f1-9ab5-edbe90d93329 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:00.004606 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-9fc0531d-607b-45f1-9ab5-edbe90d93329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:00.083327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1397/2771] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:00.091055 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716e3cc-ba08-4b71-bc7d-febf182b9c35 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:00.091410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716e3cc-ba08-4b71-bc7d-febf182b9c35 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:00.091739 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716e3cc-ba08-4b71-bc7d-febf182b9c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:00.113277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-4716e3cc-ba08-4b71-bc7d-febf182b9c35 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:00.114518 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4716e3cc-ba08-4b71-bc7d-febf182b9c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:00.122522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1375/2772] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:00.127563 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0531d-607b-45f1-9ab5-edbe90d93329 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:00.138025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0531d-607b-45f1-9ab5-edbe90d93329 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:00.142938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79a2cf-9d00-40d2-b09b-b3de662946a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:00.143553 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79a2cf-9d00-40d2-b09b-b3de662946a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:00.143833 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79a2cf-9d00-40d2-b09b-b3de662946a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:00.144208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79a2cf-9d00-40d2-b09b-b3de662946a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:00.144866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79a2cf-9d00-40d2-b09b-b3de662946a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:10:00.145168 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79a2cf-9d00-40d2-b09b-b3de662946a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:00.146771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79a2cf-9d00-40d2-b09b-b3de662946a5 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:00.159526 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1398/2773] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 06:10:00.163973 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716e3cc-ba08-4b71-bc7d-febf182b9c35 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:00.174028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716e3cc-ba08-4b71-bc7d-febf182b9c35 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:00.178864 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0c6b5-e3d7-456c-b2f5-2b43afd0a3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:00.179490 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0c6b5-e3d7-456c-b2f5-2b43afd0a3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:00.179768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0c6b5-e3d7-456c-b2f5-2b43afd0a3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:00.180077 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0c6b5-e3d7-456c-b2f5-2b43afd0a3e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:00.180875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0c6b5-e3d7-456c-b2f5-2b43afd0a3e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:00.182769 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0c6b5-e3d7-456c-b2f5-2b43afd0a3e6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:00.211392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1376/2774] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:10:00 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:10:00.215903 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79a2cf-9d00-40d2-b09b-b3de662946a5 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:00.226466 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79a2cf-9d00-40d2-b09b-b3de662946a5 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:00.281582 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:00.293742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb83054-753b-43e8-bd71-e99e0f62995c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:00.294646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb83054-753b-43e8-bd71-e99e0f62995c admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:00.295256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb83054-753b-43e8-bd71-e99e0f62995c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:00.295537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb83054-753b-43e8-bd71-e99e0f62995c admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:00.296256 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb83054-753b-43e8-bd71-e99e0f62995c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:00.299114 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb83054-753b-43e8-bd71-e99e0f62995c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:00.321543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1399/2775] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:10:00 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:10:00.325593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0c6b5-e3d7-456c-b2f5-2b43afd0a3e6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:00.336547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0c6b5-e3d7-456c-b2f5-2b43afd0a3e6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:00.341264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614ce45-f7ae-432d-a3d7-9335699f01bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:00.341879 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614ce45-f7ae-432d-a3d7-9335699f01bc admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:00.342159 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614ce45-f7ae-432d-a3d7-9335699f01bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:00.342431 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614ce45-f7ae-432d-a3d7-9335699f01bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:00.343079 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614ce45-f7ae-432d-a3d7-9335699f01bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:00.345236 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614ce45-f7ae-432d-a3d7-9335699f01bc admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:00.356053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1377/2776] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:10:00.360964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb83054-753b-43e8-bd71-e99e0f62995c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:00.371448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb83054-753b-43e8-bd71-e99e0f62995c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:00.376537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c2779-17d9-43f4-a3e1-cc57fffd6909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:00.377262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c2779-17d9-43f4-a3e1-cc57fffd6909 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:00.377680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c2779-17d9-43f4-a3e1-cc57fffd6909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:00.377853 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c2779-17d9-43f4-a3e1-cc57fffd6909 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:00.378643 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c2779-17d9-43f4-a3e1-cc57fffd6909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:00.380340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c2779-17d9-43f4-a3e1-cc57fffd6909 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:00.394950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1400/2777] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:10:00.396459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614ce45-f7ae-432d-a3d7-9335699f01bc admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:00.408198 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614ce45-f7ae-432d-a3d7-9335699f01bc admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:00.413182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef985d-1f94-48c0-b2f1-c8bb5a797e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:00.413845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef985d-1f94-48c0-b2f1-c8bb5a797e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:00.414285 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef985d-1f94-48c0-b2f1-c8bb5a797e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:00.414563 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef985d-1f94-48c0-b2f1-c8bb5a797e5e admin admin] PATH_INFO: `/v3/projects/f65e2aee89244ff8b1406853f94ffdb9/users/c8f4d6ec486e4f6bb9dd647df58e18a9/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:00.431200 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef985d-1f94-48c0-b2f1-c8bb5a797e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f65e2aee89244ff8b1406853f94ffdb9, user_id=c8f4d6ec486e4f6bb9dd647df58e18a9, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:00.433961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef985d-1f94-48c0-b2f1-c8bb5a797e5e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:00.458114 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:00.463859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1378/2778] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:10:00 2022] PUT /identity/v3/projects/f65e2aee89244ff8b1406853f94ffdb9/users/c8f4d6ec486e4f6bb9dd647df58e18a9/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:00.475499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61567c48-54e6-4656-83a3-b63e20eb54ed None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:00.475789 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61567c48-54e6-4656-83a3-b63e20eb54ed None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:00.476101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61567c48-54e6-4656-83a3-b63e20eb54ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:00.505467 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-61567c48-54e6-4656-83a3-b63e20eb54ed None None] MFA Rules not processed for user `c8f4d6ec486e4f6bb9dd647df58e18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:00.506685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-61567c48-54e6-4656-83a3-b63e20eb54ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:00.578857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1401/2779] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:00.589949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef985d-1f94-48c0-b2f1-c8bb5a797e5e admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:00.591763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-daef985d-1f94-48c0-b2f1-c8bb5a797e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:00.651052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef985d-1f94-48c0-b2f1-c8bb5a797e5e admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:00.654742 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4cc0c-3408-4ecb-b2d1-1efe9dcad753 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0330c79c391d463b91f99e030c6dcc07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:00.655357 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4cc0c-3408-4ecb-b2d1-1efe9dcad753 service glance] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:00.655677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4cc0c-3408-4ecb-b2d1-1efe9dcad753 service glance] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:00.656243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4cc0c-3408-4ecb-b2d1-1efe9dcad753 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:00.667905 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4cc0c-3408-4ecb-b2d1-1efe9dcad753 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:00.669386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4cc0c-3408-4ecb-b2d1-1efe9dcad753 service glance] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:00.679203 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1379/2780] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:10:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:10:10.305791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61567c48-54e6-4656-83a3-b63e20eb54ed None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:10.307433 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-61567c48-54e6-4656-83a3-b63e20eb54ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:10.375034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61567c48-54e6-4656-83a3-b63e20eb54ed None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:10.380492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:10.381113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:10.381388 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:10.381654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] PATH_INFO: `/v3/users/51ca25c276d24b128233d102a4c87240` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:10.382753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ca25c276d24b128233d102a4c87240)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:10.384698 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:10.434617 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ca25c276d24b128233d102a4c87240'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:10.435286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ca25c276d24b128233d102a4c87240'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:10.443314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ca25c276d24b128233d102a4c87240'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:10.451552 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ca25c276d24b128233d102a4c87240'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:10.452777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:10.466969 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ca25c276d24b128233d102a4c87240'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:10.473289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1402/2781] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:10:10 2022] DELETE /identity/v3/users/51ca25c276d24b128233d102a4c87240 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:10.477762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b59c49-07b9-45ce-9bde-24672af35247 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:10.478243 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b59c49-07b9-45ce-9bde-24672af35247 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:10.478422 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b59c49-07b9-45ce-9bde-24672af35247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:10.499623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-b6b59c49-07b9-45ce-9bde-24672af35247 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:10.502912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b6b59c49-07b9-45ce-9bde-24672af35247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:10.590704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1380/2782] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:10.600941 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:10.602791 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:10.682633 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7947d-78b6-4431-a274-87d0fde8f398 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:10.687172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee0977-c99d-41a9-b78c-ff2947b9ffa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:10.687827 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee0977-c99d-41a9-b78c-ff2947b9ffa1 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:10.688733 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee0977-c99d-41a9-b78c-ff2947b9ffa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:10.688989 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee0977-c99d-41a9-b78c-ff2947b9ffa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:10.698413 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee0977-c99d-41a9-b78c-ff2947b9ffa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:10.700199 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee0977-c99d-41a9-b78c-ff2947b9ffa1 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:10.710357 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1403/2783] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:10:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:10:11.005447 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b59c49-07b9-45ce-9bde-24672af35247 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:11.007125 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b6b59c49-07b9-45ce-9bde-24672af35247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:11.031371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b59c49-07b9-45ce-9bde-24672af35247 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:11.039575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefebdb7-142d-496b-b977-f00911e7c731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:11.040416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefebdb7-142d-496b-b977-f00911e7c731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:11.060242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefebdb7-142d-496b-b977-f00911e7c731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:11.060242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefebdb7-142d-496b-b977-f00911e7c731 admin admin] PATH_INFO: `/v3/projects/54eb11eb37d74f6dbd22409bdca6f82b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:11.060242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefebdb7-142d-496b-b977-f00911e7c731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54eb11eb37d74f6dbd22409bdca6f82b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:11.060242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefebdb7-142d-496b-b977-f00911e7c731 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:11.103331 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-aefebdb7-142d-496b-b977-f00911e7c731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54eb11eb37d74f6dbd22409bdca6f82b', 'request_id': 'req-aefebdb7-142d-496b-b977-f00911e7c731'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:11.111906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-aefebdb7-142d-496b-b977-f00911e7c731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54eb11eb37d74f6dbd22409bdca6f82b', 'request_id': 'req-aefebdb7-142d-496b-b977-f00911e7c731'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:11.118612 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-aefebdb7-142d-496b-b977-f00911e7c731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54eb11eb37d74f6dbd22409bdca6f82b', 'request_id': 'req-aefebdb7-142d-496b-b977-f00911e7c731'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:11.129759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-aefebdb7-142d-496b-b977-f00911e7c731 admin admin] The token cache is being invalidate because project 54eb11eb37d74f6dbd22409bdca6f82b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:10:11.130050 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-aefebdb7-142d-496b-b977-f00911e7c731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:11.130701 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-aefebdb7-142d-496b-b977-f00911e7c731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:11.132547 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1381/2784] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:10:11 2022] DELETE /identity/v3/projects/54eb11eb37d74f6dbd22409bdca6f82b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:11.137316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bda10-02bc-4c59-b246-2e18dd435bf7 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:11.137598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bda10-02bc-4c59-b246-2e18dd435bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:11.137876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bda10-02bc-4c59-b246-2e18dd435bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:11.159668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-320bda10-02bc-4c59-b246-2e18dd435bf7 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:11.160951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-320bda10-02bc-4c59-b246-2e18dd435bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:11.223177 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1404/2785] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:11.230144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b4c55-0f8e-4819-ba5e-f3c4ebf59afa None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:11.230482 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b4c55-0f8e-4819-ba5e-f3c4ebf59afa None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:11.230763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b4c55-0f8e-4819-ba5e-f3c4ebf59afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:11.250427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f73b4c55-0f8e-4819-ba5e-f3c4ebf59afa None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:11.251660 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f73b4c55-0f8e-4819-ba5e-f3c4ebf59afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:11.259529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1382/2786] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:10:11.263794 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bda10-02bc-4c59-b246-2e18dd435bf7 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:11.274598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bda10-02bc-4c59-b246-2e18dd435bf7 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:11.279275 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d35b0-27cd-4659-8e23-82038754083e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:11.279886 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d35b0-27cd-4659-8e23-82038754083e admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:11.280279 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d35b0-27cd-4659-8e23-82038754083e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:11.280560 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d35b0-27cd-4659-8e23-82038754083e admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:11.281207 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d35b0-27cd-4659-8e23-82038754083e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:10:11.281509 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d35b0-27cd-4659-8e23-82038754083e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:11.283172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d35b0-27cd-4659-8e23-82038754083e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:11.294906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1405/2787] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:10:11.300035 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b4c55-0f8e-4819-ba5e-f3c4ebf59afa None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:11.310081 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b4c55-0f8e-4819-ba5e-f3c4ebf59afa None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:11.316195 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ba00a-6f07-4fb8-b21a-6486662fbce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:11.316823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ba00a-6f07-4fb8-b21a-6486662fbce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:11.317102 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ba00a-6f07-4fb8-b21a-6486662fbce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:11.317374 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ba00a-6f07-4fb8-b21a-6486662fbce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:11.318150 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ba00a-6f07-4fb8-b21a-6486662fbce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:11.320010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ba00a-6f07-4fb8-b21a-6486662fbce9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:11.342742 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:11.352034 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1383/2788] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:10:11 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:10:11.355870 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d35b0-27cd-4659-8e23-82038754083e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:11.366785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d35b0-27cd-4659-8e23-82038754083e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:11.425059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7d36b-e848-4940-b13c-655b50733ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:11.425667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7d36b-e848-4940-b13c-655b50733ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:11.425946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7d36b-e848-4940-b13c-655b50733ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:11.426218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7d36b-e848-4940-b13c-655b50733ed0 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:11.426847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7d36b-e848-4940-b13c-655b50733ed0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:11.428817 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7d36b-e848-4940-b13c-655b50733ed0 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:11.446180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:11.451812 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1406/2789] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:10:11 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:10:11.455685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ba00a-6f07-4fb8-b21a-6486662fbce9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:11.466406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ba00a-6f07-4fb8-b21a-6486662fbce9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:11.471422 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b47bd-dac2-42ce-bd73-843ea075d820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:11.472086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b47bd-dac2-42ce-bd73-843ea075d820 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:11.472369 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b47bd-dac2-42ce-bd73-843ea075d820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:11.472643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b47bd-dac2-42ce-bd73-843ea075d820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:11.473286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b47bd-dac2-42ce-bd73-843ea075d820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:11.474877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b47bd-dac2-42ce-bd73-843ea075d820 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:11.485462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1384/2790] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:10:11.489574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7d36b-e848-4940-b13c-655b50733ed0 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:11.499567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7d36b-e848-4940-b13c-655b50733ed0 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:11.505126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e3f59-d501-4b4f-b3d9-972b7f34ded2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:11.505811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e3f59-d501-4b4f-b3d9-972b7f34ded2 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:11.506098 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e3f59-d501-4b4f-b3d9-972b7f34ded2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:11.506376 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e3f59-d501-4b4f-b3d9-972b7f34ded2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:11.507020 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e3f59-d501-4b4f-b3d9-972b7f34ded2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:11.508652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e3f59-d501-4b4f-b3d9-972b7f34ded2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:11.520096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1407/2791] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:10:11.524568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b47bd-dac2-42ce-bd73-843ea075d820 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:11.535156 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b47bd-dac2-42ce-bd73-843ea075d820 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:11.540487 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc3aca-5b96-4448-848d-8cf7da002468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:11.541146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc3aca-5b96-4448-848d-8cf7da002468 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:11.541425 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc3aca-5b96-4448-848d-8cf7da002468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:11.541703 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc3aca-5b96-4448-848d-8cf7da002468 admin admin] PATH_INFO: `/v3/projects/aa3af5a9e2fd4320bff58a90beea56f3/users/dbc8fc59f299496ca3f11520f23b3fd3/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:11.557143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc3aca-5b96-4448-848d-8cf7da002468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3af5a9e2fd4320bff58a90beea56f3, user_id=dbc8fc59f299496ca3f11520f23b3fd3, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:11.559801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc3aca-5b96-4448-848d-8cf7da002468 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:11.583776 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1385/2792] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:10:11 2022] PUT /identity/v3/projects/aa3af5a9e2fd4320bff58a90beea56f3/users/dbc8fc59f299496ca3f11520f23b3fd3/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:11.596430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170a57f-384e-4d0d-9498-4bb58ff8ad30 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:11.596753 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170a57f-384e-4d0d-9498-4bb58ff8ad30 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:11.597128 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170a57f-384e-4d0d-9498-4bb58ff8ad30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:11.626935 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c170a57f-384e-4d0d-9498-4bb58ff8ad30 None None] MFA Rules not processed for user `dbc8fc59f299496ca3f11520f23b3fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:11.628187 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c170a57f-384e-4d0d-9498-4bb58ff8ad30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:11.696421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1408/2793] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:11.737806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc3aca-5b96-4448-848d-8cf7da002468 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:11.739519 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6cfc3aca-5b96-4448-848d-8cf7da002468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:11.807391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc3aca-5b96-4448-848d-8cf7da002468 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:11.811106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09aad09-1b73-4282-bae9-708556b36189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:11.811721 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aad09-1b73-4282-bae9-708556b36189 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:11.812032 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aad09-1b73-4282-bae9-708556b36189 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:11.812304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09aad09-1b73-4282-bae9-708556b36189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:11.821858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09aad09-1b73-4282-bae9-708556b36189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:11.823293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09aad09-1b73-4282-bae9-708556b36189 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:11.843835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1386/2794] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:10:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:10:11.862290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170a57f-384e-4d0d-9498-4bb58ff8ad30 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:11.926654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170a57f-384e-4d0d-9498-4bb58ff8ad30 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:11.931219 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:11.931846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:11.932161 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:11.932437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] PATH_INFO: `/v3/users/dbc8fc59f299496ca3f11520f23b3fd3` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:11.933567 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbc8fc59f299496ca3f11520f23b3fd3)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:11.935492 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:11.982831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbc8fc59f299496ca3f11520f23b3fd3'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:11.983536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbc8fc59f299496ca3f11520f23b3fd3'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:11.991406 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbc8fc59f299496ca3f11520f23b3fd3'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:11.999138 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbc8fc59f299496ca3f11520f23b3fd3'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:12.012118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbc8fc59f299496ca3f11520f23b3fd3'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:12.013686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:12.020160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1409/2795] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:10:11 2022] DELETE /identity/v3/users/dbc8fc59f299496ca3f11520f23b3fd3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:12.025125 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da899a73-6683-47bb-ab95-7456a2cdcf95 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:12.025412 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da899a73-6683-47bb-ab95-7456a2cdcf95 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:12.025685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da899a73-6683-47bb-ab95-7456a2cdcf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:12.045620 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-da899a73-6683-47bb-ab95-7456a2cdcf95 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:12.047021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-da899a73-6683-47bb-ab95-7456a2cdcf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:12.119416 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1387/2796] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:12.130044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:12.131516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:12.194374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164d715-9738-43a8-8135-e422490b8146 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:12.197972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0027fae-93a7-411c-88c3-6317e6a53a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:12.198572 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0027fae-93a7-411c-88c3-6317e6a53a2b service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:12.198852 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0027fae-93a7-411c-88c3-6317e6a53a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:12.199147 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0027fae-93a7-411c-88c3-6317e6a53a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:12.208497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0027fae-93a7-411c-88c3-6317e6a53a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:12.209918 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0027fae-93a7-411c-88c3-6317e6a53a2b service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:12.219336 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1410/2797] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:10:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:10:12.509468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da899a73-6683-47bb-ab95-7456a2cdcf95 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:12.510973 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-da899a73-6683-47bb-ab95-7456a2cdcf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:12.526437 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da899a73-6683-47bb-ab95-7456a2cdcf95 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:12.531264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea99669-045b-4bf2-abb4-27b3bbee011e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:12.533411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea99669-045b-4bf2-abb4-27b3bbee011e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:12.534023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea99669-045b-4bf2-abb4-27b3bbee011e admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:12.534326 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea99669-045b-4bf2-abb4-27b3bbee011e admin admin] PATH_INFO: `/v3/projects/aa3af5a9e2fd4320bff58a90beea56f3` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:12.535490 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea99669-045b-4bf2-abb4-27b3bbee011e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa3af5a9e2fd4320bff58a90beea56f3)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:12.537564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea99669-045b-4bf2-abb4-27b3bbee011e admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:12.589519 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4ea99669-045b-4bf2-abb4-27b3bbee011e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa3af5a9e2fd4320bff58a90beea56f3', 'request_id': 'req-4ea99669-045b-4bf2-abb4-27b3bbee011e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:12.599009 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4ea99669-045b-4bf2-abb4-27b3bbee011e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa3af5a9e2fd4320bff58a90beea56f3', 'request_id': 'req-4ea99669-045b-4bf2-abb4-27b3bbee011e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:12.606721 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4ea99669-045b-4bf2-abb4-27b3bbee011e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa3af5a9e2fd4320bff58a90beea56f3', 'request_id': 'req-4ea99669-045b-4bf2-abb4-27b3bbee011e'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:12.608313 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:12.623957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4ea99669-045b-4bf2-abb4-27b3bbee011e admin admin] The token cache is being invalidate because project aa3af5a9e2fd4320bff58a90beea56f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:10:12.624509 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4ea99669-045b-4bf2-abb4-27b3bbee011e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:12.626545 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4ea99669-045b-4bf2-abb4-27b3bbee011e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:12.634345 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1388/2798] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:10:12 2022] DELETE /identity/v3/projects/aa3af5a9e2fd4320bff58a90beea56f3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:12.637439 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732005b-7ba0-4c14-aed5-b4aba4dfc41a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:12.637979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732005b-7ba0-4c14-aed5-b4aba4dfc41a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:12.638249 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732005b-7ba0-4c14-aed5-b4aba4dfc41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:12.668241 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8732005b-7ba0-4c14-aed5-b4aba4dfc41a None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:12.669555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8732005b-7ba0-4c14-aed5-b4aba4dfc41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:12.738459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1411/2799] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:12.746398 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054b214-d3db-4cb5-9b4f-aa6feda95b53 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:12.746684 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054b214-d3db-4cb5-9b4f-aa6feda95b53 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:12.746957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054b214-d3db-4cb5-9b4f-aa6feda95b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:12.783237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-a054b214-d3db-4cb5-9b4f-aa6feda95b53 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:12.784531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-a054b214-d3db-4cb5-9b4f-aa6feda95b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:12.808020 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1389/2800] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:12.809442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732005b-7ba0-4c14-aed5-b4aba4dfc41a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:12.821785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732005b-7ba0-4c14-aed5-b4aba4dfc41a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:12.826978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf3657-51e1-41ab-a3eb-57abbf131c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:12.827595 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf3657-51e1-41ab-a3eb-57abbf131c07 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:12.827940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf3657-51e1-41ab-a3eb-57abbf131c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:12.828304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf3657-51e1-41ab-a3eb-57abbf131c07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:12.829725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf3657-51e1-41ab-a3eb-57abbf131c07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:10:12.830030 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf3657-51e1-41ab-a3eb-57abbf131c07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:12.831755 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf3657-51e1-41ab-a3eb-57abbf131c07 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:12.848550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1412/2801] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:10:12.849821 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054b214-d3db-4cb5-9b4f-aa6feda95b53 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:12.859406 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054b214-d3db-4cb5-9b4f-aa6feda95b53 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:12.865363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7de05-15ea-4da2-88db-4574e4962154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:12.865975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7de05-15ea-4da2-88db-4574e4962154 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:12.866285 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7de05-15ea-4da2-88db-4574e4962154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:12.866685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7de05-15ea-4da2-88db-4574e4962154 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:12.867371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa7de05-15ea-4da2-88db-4574e4962154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:12.869463 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa7de05-15ea-4da2-88db-4574e4962154 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:12.911301 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1390/2802] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:10:12 2022] POST /identity/v3/projects => generated 391 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:10:12.915570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf3657-51e1-41ab-a3eb-57abbf131c07 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:12.927055 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf3657-51e1-41ab-a3eb-57abbf131c07 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:12.987029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ba783-c42b-461d-9865-281c1b646a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:12.987652 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ba783-c42b-461d-9865-281c1b646a6e admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:12.988584 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ba783-c42b-461d-9865-281c1b646a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:12.988584 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ba783-c42b-461d-9865-281c1b646a6e admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:12.988927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ba783-c42b-461d-9865-281c1b646a6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:12.991273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ba783-c42b-461d-9865-281c1b646a6e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:13.014916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1413/2803] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:10:12 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:10:13.019610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7de05-15ea-4da2-88db-4574e4962154 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:13.035790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7de05-15ea-4da2-88db-4574e4962154 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:13.040514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9a8ea-a962-4aec-9730-2fdcc5f98f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:13.041162 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9a8ea-a962-4aec-9730-2fdcc5f98f9d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:13.042008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9a8ea-a962-4aec-9730-2fdcc5f98f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:13.042008 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9a8ea-a962-4aec-9730-2fdcc5f98f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:13.042336 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9a8ea-a962-4aec-9730-2fdcc5f98f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:13.043935 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9a8ea-a962-4aec-9730-2fdcc5f98f9d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:13.055223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1391/2804] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:10:13.059316 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ba783-c42b-461d-9865-281c1b646a6e admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:13.071947 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ba783-c42b-461d-9865-281c1b646a6e admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:13.080958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6f3d6-b6a8-436f-a28c-d28287a16da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:13.081695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6f3d6-b6a8-436f-a28c-d28287a16da5 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:13.081987 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6f3d6-b6a8-436f-a28c-d28287a16da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:13.082262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6f3d6-b6a8-436f-a28c-d28287a16da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:13.082914 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6f3d6-b6a8-436f-a28c-d28287a16da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:13.084555 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6f3d6-b6a8-436f-a28c-d28287a16da5 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:13.096129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1414/2805] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:10:13.100484 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9a8ea-a962-4aec-9730-2fdcc5f98f9d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:13.111084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9a8ea-a962-4aec-9730-2fdcc5f98f9d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:13.115929 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb50ee-223e-43f2-8a55-8298538dc576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:13.116619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb50ee-223e-43f2-8a55-8298538dc576 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:13.116938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb50ee-223e-43f2-8a55-8298538dc576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:13.117216 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb50ee-223e-43f2-8a55-8298538dc576 admin admin] PATH_INFO: `/v3/projects/4b8174e441564ea5b8fd212b9b6fb31e/users/6db14b03752c43eb8d8bfc2305c60d4f/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:13.144528 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb50ee-223e-43f2-8a55-8298538dc576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b8174e441564ea5b8fd212b9b6fb31e, user_id=6db14b03752c43eb8d8bfc2305c60d4f, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:13.148092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb50ee-223e-43f2-8a55-8298538dc576 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:13.175331 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1392/2806] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:10:13 2022] PUT /identity/v3/projects/4b8174e441564ea5b8fd212b9b6fb31e/users/6db14b03752c43eb8d8bfc2305c60d4f/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:13.185866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8939c-2dd0-4206-b574-e385f20b3692 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:13.186153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8939c-2dd0-4206-b574-e385f20b3692 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:13.186429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8939c-2dd0-4206-b574-e385f20b3692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:13.208897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:13.226238 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-02f8939c-2dd0-4206-b574-e385f20b3692 None None] MFA Rules not processed for user `6db14b03752c43eb8d8bfc2305c60d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:13.227460 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-02f8939c-2dd0-4206-b574-e385f20b3692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:13.309520 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1415/2807] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:13.321086 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb50ee-223e-43f2-8a55-8298538dc576 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:13.323569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e9fb50ee-223e-43f2-8a55-8298538dc576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:13.384384 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb50ee-223e-43f2-8a55-8298538dc576 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:13.388040 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e323ec-7937-499e-8177-a02e8f80bc95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:13.388643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e323ec-7937-499e-8177-a02e8f80bc95 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:13.388919 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e323ec-7937-499e-8177-a02e8f80bc95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:13.389223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e323ec-7937-499e-8177-a02e8f80bc95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:13.399981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e323ec-7937-499e-8177-a02e8f80bc95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:13.401466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e323ec-7937-499e-8177-a02e8f80bc95 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:13.411184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1393/2808] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:10:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:10:23.310938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=29 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:23.406497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=2,keystone:SELECT=56,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:44.297625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8939c-2dd0-4206-b574-e385f20b3692 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:44.359854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8939c-2dd0-4206-b574-e385f20b3692 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:44.364380 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:44.365014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:44.365288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:44.365561 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] PATH_INFO: `/v3/users/6db14b03752c43eb8d8bfc2305c60d4f` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:44.366690 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6db14b03752c43eb8d8bfc2305c60d4f)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:44.368653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:44.417283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6db14b03752c43eb8d8bfc2305c60d4f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:44.418049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6db14b03752c43eb8d8bfc2305c60d4f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:44.425226 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6db14b03752c43eb8d8bfc2305c60d4f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:44.435776 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6db14b03752c43eb8d8bfc2305c60d4f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:44.448528 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6db14b03752c43eb8d8bfc2305c60d4f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:44.454956 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1416/2809] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:10:44 2022] DELETE /identity/v3/users/6db14b03752c43eb8d8bfc2305c60d4f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:10:44.459474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bff4cb-1b7f-4056-8381-a91b27a581b3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:44.460139 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bff4cb-1b7f-4056-8381-a91b27a581b3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:44.460139 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bff4cb-1b7f-4056-8381-a91b27a581b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:44.479194 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-f2bff4cb-1b7f-4056-8381-a91b27a581b3 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:44.480426 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f2bff4cb-1b7f-4056-8381-a91b27a581b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:44.543657 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1394/2810] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:10:44.553958 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:44.555386 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:44.614629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c07ae4-3ec3-4fb9-b28a-df895ea0d929 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:44.618295 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a257ed-9e63-4a1b-8f84-c4e2044304ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:44.618902 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a257ed-9e63-4a1b-8f84-c4e2044304ba service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:44.619183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a257ed-9e63-4a1b-8f84-c4e2044304ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:44.619459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a257ed-9e63-4a1b-8f84-c4e2044304ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:44.630397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a257ed-9e63-4a1b-8f84-c4e2044304ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:44.631841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a257ed-9e63-4a1b-8f84-c4e2044304ba service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:44.642337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1417/2811] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:10:44.933988 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bff4cb-1b7f-4056-8381-a91b27a581b3 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:44.936413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-f2bff4cb-1b7f-4056-8381-a91b27a581b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:44.950170 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bff4cb-1b7f-4056-8381-a91b27a581b3 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:44.955561 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20964718-7815-4c89-9e5f-c35eec12b4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:44.956252 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20964718-7815-4c89-9e5f-c35eec12b4dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:44.956560 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20964718-7815-4c89-9e5f-c35eec12b4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:44.956763 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20964718-7815-4c89-9e5f-c35eec12b4dd admin admin] PATH_INFO: `/v3/projects/4b8174e441564ea5b8fd212b9b6fb31e` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:44.958112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20964718-7815-4c89-9e5f-c35eec12b4dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b8174e441564ea5b8fd212b9b6fb31e)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:44.959938 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20964718-7815-4c89-9e5f-c35eec12b4dd admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:45.012149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-20964718-7815-4c89-9e5f-c35eec12b4dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b8174e441564ea5b8fd212b9b6fb31e', 'request_id': 'req-20964718-7815-4c89-9e5f-c35eec12b4dd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:45.021149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-20964718-7815-4c89-9e5f-c35eec12b4dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b8174e441564ea5b8fd212b9b6fb31e', 'request_id': 'req-20964718-7815-4c89-9e5f-c35eec12b4dd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:45.028219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-20964718-7815-4c89-9e5f-c35eec12b4dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b8174e441564ea5b8fd212b9b6fb31e', 'request_id': 'req-20964718-7815-4c89-9e5f-c35eec12b4dd'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:45.039611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-20964718-7815-4c89-9e5f-c35eec12b4dd admin admin] The token cache is being invalidate because project 4b8174e441564ea5b8fd212b9b6fb31e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:10:45.039905 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-20964718-7815-4c89-9e5f-c35eec12b4dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:45.040670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-20964718-7815-4c89-9e5f-c35eec12b4dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:45.042322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1395/2812] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:10:44 2022] DELETE /identity/v3/projects/4b8174e441564ea5b8fd212b9b6fb31e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:45.047476 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f2308-1dd1-4f5a-b69e-5058e8c416ac None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:45.047739 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f2308-1dd1-4f5a-b69e-5058e8c416ac None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:45.048043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f2308-1dd1-4f5a-b69e-5058e8c416ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:45.068172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a96f2308-1dd1-4f5a-b69e-5058e8c416ac None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:45.069415 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a96f2308-1dd1-4f5a-b69e-5058e8c416ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:45.089417 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:45.138972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1418/2813] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:45.149260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ef820-862e-4a34-83f2-0654a1575ff4 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:45.149552 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ef820-862e-4a34-83f2-0654a1575ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:45.149832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ef820-862e-4a34-83f2-0654a1575ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:45.170796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-272ef820-862e-4a34-83f2-0654a1575ff4 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:45.172078 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-272ef820-862e-4a34-83f2-0654a1575ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:45.180468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1396/2814] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:45.186549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f2308-1dd1-4f5a-b69e-5058e8c416ac None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:45.200501 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f2308-1dd1-4f5a-b69e-5058e8c416ac None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:45.205997 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef45a3-0653-4755-8c8b-ff544bf08029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:45.206593 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef45a3-0653-4755-8c8b-ff544bf08029 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:45.206873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef45a3-0653-4755-8c8b-ff544bf08029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:45.207179 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef45a3-0653-4755-8c8b-ff544bf08029 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:45.207950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef45a3-0653-4755-8c8b-ff544bf08029 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:10:45.208212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef45a3-0653-4755-8c8b-ff544bf08029 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:45.209796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef45a3-0653-4755-8c8b-ff544bf08029 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:45.223250 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1419/2815] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:10:45.228258 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ef820-862e-4a34-83f2-0654a1575ff4 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:45.238543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ef820-862e-4a34-83f2-0654a1575ff4 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:45.243242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f8ffe-683c-4689-8385-4544c3ab38ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:45.243847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f8ffe-683c-4689-8385-4544c3ab38ef admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:45.244175 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f8ffe-683c-4689-8385-4544c3ab38ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:45.244449 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f8ffe-683c-4689-8385-4544c3ab38ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:45.245215 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f8ffe-683c-4689-8385-4544c3ab38ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:45.247251 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f8ffe-683c-4689-8385-4544c3ab38ef admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:45.277054 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1397/2816] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:10:45 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:10:45.282508 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef45a3-0653-4755-8c8b-ff544bf08029 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:45.293124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef45a3-0653-4755-8c8b-ff544bf08029 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:45.353235 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bea704-5aa7-4827-b68f-962e8c33fa3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:45.353906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bea704-5aa7-4827-b68f-962e8c33fa3b admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:45.354107 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bea704-5aa7-4827-b68f-962e8c33fa3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:45.354382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bea704-5aa7-4827-b68f-962e8c33fa3b admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:45.355037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bea704-5aa7-4827-b68f-962e8c33fa3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:45.357051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bea704-5aa7-4827-b68f-962e8c33fa3b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:45.380819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1420/2817] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:10:45 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:10:45.384971 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f8ffe-683c-4689-8385-4544c3ab38ef admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:45.395955 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f8ffe-683c-4689-8385-4544c3ab38ef admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:45.400805 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa408ddc-ae55-470b-9a3a-5d9ba69eba18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:45.401400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa408ddc-ae55-470b-9a3a-5d9ba69eba18 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:45.401676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa408ddc-ae55-470b-9a3a-5d9ba69eba18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:45.401946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa408ddc-ae55-470b-9a3a-5d9ba69eba18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:45.402581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa408ddc-ae55-470b-9a3a-5d9ba69eba18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:45.404198 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa408ddc-ae55-470b-9a3a-5d9ba69eba18 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:45.414524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1398/2818] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:10:45.418653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bea704-5aa7-4827-b68f-962e8c33fa3b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:45.429310 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bea704-5aa7-4827-b68f-962e8c33fa3b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:45.433828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2b6d1-12aa-43f3-a2e7-514a535d3b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:45.434414 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2b6d1-12aa-43f3-a2e7-514a535d3b3d admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:45.434692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2b6d1-12aa-43f3-a2e7-514a535d3b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:45.434957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2b6d1-12aa-43f3-a2e7-514a535d3b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:45.435583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2b6d1-12aa-43f3-a2e7-514a535d3b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:45.437873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2b6d1-12aa-43f3-a2e7-514a535d3b3d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:45.448532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1421/2819] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:10:45.453445 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa408ddc-ae55-470b-9a3a-5d9ba69eba18 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:45.464132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa408ddc-ae55-470b-9a3a-5d9ba69eba18 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:45.469462 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342007d-b6b6-4256-8379-283d0b29f1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:45.470149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342007d-b6b6-4256-8379-283d0b29f1d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:45.470432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342007d-b6b6-4256-8379-283d0b29f1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:45.470708 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342007d-b6b6-4256-8379-283d0b29f1d6 admin admin] PATH_INFO: `/v3/projects/10427ce4a1644db89943273919d296cc/users/309d1efd676845bbaa30f472b19ffa34/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:45.486637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342007d-b6b6-4256-8379-283d0b29f1d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10427ce4a1644db89943273919d296cc, user_id=309d1efd676845bbaa30f472b19ffa34, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:45.489474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342007d-b6b6-4256-8379-283d0b29f1d6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:45.504898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1399/2820] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:10:45 2022] PUT /identity/v3/projects/10427ce4a1644db89943273919d296cc/users/309d1efd676845bbaa30f472b19ffa34/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:45.509934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8211c-ce09-4974-a1d3-b929ec197482 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:45.510260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8211c-ce09-4974-a1d3-b929ec197482 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:45.510540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8211c-ce09-4974-a1d3-b929ec197482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:45.530984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-5ed8211c-ce09-4974-a1d3-b929ec197482 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:45.532278 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5ed8211c-ce09-4974-a1d3-b929ec197482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:45.595538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1422/2821] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:45.605779 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342007d-b6b6-4256-8379-283d0b29f1d6 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:45.607442 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-9342007d-b6b6-4256-8379-283d0b29f1d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:45.632994 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:45.674746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342007d-b6b6-4256-8379-283d0b29f1d6 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:45.678524 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c6433-cccc-4159-8827-38816596042b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:45.679221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c6433-cccc-4159-8827-38816596042b service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:45.679575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c6433-cccc-4159-8827-38816596042b service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:45.679915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c6433-cccc-4159-8827-38816596042b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:45.690097 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c6433-cccc-4159-8827-38816596042b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:45.691596 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c6433-cccc-4159-8827-38816596042b service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:45.701554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1400/2822] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:10:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:10:50.191491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284cce8-d1cf-4ba5-ba81-aedc73c44b11 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:50.191961 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284cce8-d1cf-4ba5-ba81-aedc73c44b11 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:50.192163 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284cce8-d1cf-4ba5-ba81-aedc73c44b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:50.235034 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2284cce8-d1cf-4ba5-ba81-aedc73c44b11 None None] MFA Rules not processed for user `309d1efd676845bbaa30f472b19ffa34`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:50.236427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2284cce8-d1cf-4ba5-ba81-aedc73c44b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:50.256217 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:50.339773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1423/2823] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:50.344280 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c6433-cccc-4159-8827-38816596042b service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:50.354908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c6433-cccc-4159-8827-38816596042b service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:50.360841 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c1b7c-f9a1-43ba-b5bc-e6997ab74c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:50.361450 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c1b7c-f9a1-43ba-b5bc-e6997ab74c31 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:50.361734 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c1b7c-f9a1-43ba-b5bc-e6997ab74c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:50.362010 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c1b7c-f9a1-43ba-b5bc-e6997ab74c31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:50.362648 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c1b7c-f9a1-43ba-b5bc-e6997ab74c31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:50.364758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c1b7c-f9a1-43ba-b5bc-e6997ab74c31 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:50.398549 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1401/2824] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:10:50 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:10:50.402431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284cce8-d1cf-4ba5-ba81-aedc73c44b11 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:50.414337 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284cce8-d1cf-4ba5-ba81-aedc73c44b11 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:50.496832 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042354f2-e74f-472a-af8e-1698b950ce11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:50.497377 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042354f2-e74f-472a-af8e-1698b950ce11 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:50.497663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042354f2-e74f-472a-af8e-1698b950ce11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:50.497934 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042354f2-e74f-472a-af8e-1698b950ce11 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:50.498680 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042354f2-e74f-472a-af8e-1698b950ce11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:50.500794 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042354f2-e74f-472a-af8e-1698b950ce11 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:50.528155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1424/2825] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:10:50 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:10:50.530168 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c1b7c-f9a1-43ba-b5bc-e6997ab74c31 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:50.544088 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c1b7c-f9a1-43ba-b5bc-e6997ab74c31 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:50.553593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a1374-35bb-4294-95d6-6e5297534e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:50.554317 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a1374-35bb-4294-95d6-6e5297534e1c admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:50.554674 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a1374-35bb-4294-95d6-6e5297534e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:50.555100 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a1374-35bb-4294-95d6-6e5297534e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:50.555743 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a1374-35bb-4294-95d6-6e5297534e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:50.557510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a1374-35bb-4294-95d6-6e5297534e1c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:50.579225 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042354f2-e74f-472a-af8e-1698b950ce11 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:50.587900 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1402/2826] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:10:50.601451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042354f2-e74f-472a-af8e-1698b950ce11 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:50.609546 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41e005-5efe-4caa-bf90-5e63ef2dba8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:50.610925 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41e005-5efe-4caa-bf90-5e63ef2dba8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:50.611412 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41e005-5efe-4caa-bf90-5e63ef2dba8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:50.619789 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41e005-5efe-4caa-bf90-5e63ef2dba8a admin admin] PATH_INFO: `/v3/projects/6c69561786614f9ebb42a36816416489/users/2bae12693aac49a2b63d13cf23cfbd3d/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:50.651259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41e005-5efe-4caa-bf90-5e63ef2dba8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c69561786614f9ebb42a36816416489, user_id=2bae12693aac49a2b63d13cf23cfbd3d, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:50.657484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41e005-5efe-4caa-bf90-5e63ef2dba8a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:50.678647 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1425/2827] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:10:50 2022] PUT /identity/v3/projects/6c69561786614f9ebb42a36816416489/users/2bae12693aac49a2b63d13cf23cfbd3d/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:50.679823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a1374-35bb-4294-95d6-6e5297534e1c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:50.695446 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a1374-35bb-4294-95d6-6e5297534e1c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:50.749559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c0840-f790-41a4-baa2-234f447b6d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:50.750392 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c0840-f790-41a4-baa2-234f447b6d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:50.750673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c0840-f790-41a4-baa2-234f447b6d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:50.750950 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c0840-f790-41a4-baa2-234f447b6d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:50.751593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c0840-f790-41a4-baa2-234f447b6d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:50.753236 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c0840-f790-41a4-baa2-234f447b6d6a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:50.763290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1403/2828] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:10:50.767473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41e005-5efe-4caa-bf90-5e63ef2dba8a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:50.777906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41e005-5efe-4caa-bf90-5e63ef2dba8a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:50.783399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad58fe-8c94-44f4-b533-e0be314692cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:50.784044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad58fe-8c94-44f4-b533-e0be314692cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:50.784313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad58fe-8c94-44f4-b533-e0be314692cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:50.784583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad58fe-8c94-44f4-b533-e0be314692cd admin admin] PATH_INFO: `/v3/projects/6c69561786614f9ebb42a36816416489/users/2bae12693aac49a2b63d13cf23cfbd3d/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:50.786497 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad58fe-8c94-44f4-b533-e0be314692cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c69561786614f9ebb42a36816416489, user_id=2bae12693aac49a2b63d13cf23cfbd3d, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:50.789211 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad58fe-8c94-44f4-b533-e0be314692cd admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:50.801484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1426/2829] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:10:50 2022] PUT /identity/v3/projects/6c69561786614f9ebb42a36816416489/users/2bae12693aac49a2b63d13cf23cfbd3d/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:55.600288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c0840-f790-41a4-baa2-234f447b6d6a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:55.602361 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ed0c0840-f790-41a4-baa2-234f447b6d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:55.630610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c0840-f790-41a4-baa2-234f447b6d6a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:55.934359 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:55.936436 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:55.937364 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:55.937753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] PATH_INFO: `/v3/users/c8f4d6ec486e4f6bb9dd647df58e18a9` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:55.944744 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8f4d6ec486e4f6bb9dd647df58e18a9)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:55.950584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:56.084106 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:56.129645 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8f4d6ec486e4f6bb9dd647df58e18a9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:56.152067 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8f4d6ec486e4f6bb9dd647df58e18a9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:56.200378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8f4d6ec486e4f6bb9dd647df58e18a9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:56.204936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8f4d6ec486e4f6bb9dd647df58e18a9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:56.208409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8f4d6ec486e4f6bb9dd647df58e18a9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:56.257223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1404/2830] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:10:55 2022] DELETE /identity/v3/users/c8f4d6ec486e4f6bb9dd647df58e18a9 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:56.261587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e330d-e8b1-4a6e-ad8b-ee119c1f9773 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:56.262022 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e330d-e8b1-4a6e-ad8b-ee119c1f9773 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:56.262190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e330d-e8b1-4a6e-ad8b-ee119c1f9773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:56.293878 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-e20e330d-e8b1-4a6e-ad8b-ee119c1f9773 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:56.296547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-e20e330d-e8b1-4a6e-ad8b-ee119c1f9773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:56.439578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1427/2831] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:56.440570 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:56.441962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:56.544785 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05781c68-579a-42f6-be08-0f929239d84a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:56.550253 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20df9b-50dc-4689-a30c-9049f021e1c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:56.551090 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20df9b-50dc-4689-a30c-9049f021e1c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:56.551382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20df9b-50dc-4689-a30c-9049f021e1c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:56.551661 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20df9b-50dc-4689-a30c-9049f021e1c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:56.571490 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20df9b-50dc-4689-a30c-9049f021e1c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:56.573776 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20df9b-50dc-4689-a30c-9049f021e1c7 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:56.594659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1405/2832] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:10:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:10:56.750723 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4c038-e099-4e74-909a-28c5ee04cb6f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:56.751068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4c038-e099-4e74-909a-28c5ee04cb6f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:56.751461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4c038-e099-4e74-909a-28c5ee04cb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:56.806876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:56.826832 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-3ad4c038-e099-4e74-909a-28c5ee04cb6f None None] MFA Rules not processed for user `2bae12693aac49a2b63d13cf23cfbd3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:56.828868 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3ad4c038-e099-4e74-909a-28c5ee04cb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:56.951346 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1428/2833] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:56.975035 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20df9b-50dc-4689-a30c-9049f021e1c7 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:56.978143 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ef20df9b-50dc-4689-a30c-9049f021e1c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:57.086721 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20df9b-50dc-4689-a30c-9049f021e1c7 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:57.088834 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4c038-e099-4e74-909a-28c5ee04cb6f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:57.090258 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3ad4c038-e099-4e74-909a-28c5ee04cb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:57.100636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9a02a-a125-499b-98db-360cbe258f03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:57.101300 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9a02a-a125-499b-98db-360cbe258f03 service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:57.101588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9a02a-a125-499b-98db-360cbe258f03 service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:57.101862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9a02a-a125-499b-98db-360cbe258f03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:57.110122 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6ff9a02a-a125-499b-98db-360cbe258f03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:57.116220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4c038-e099-4e74-909a-28c5ee04cb6f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:57.122189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82face-d287-40a3-b647-a57c001931d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:57.123059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82face-d287-40a3-b647-a57c001931d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:57.123376 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82face-d287-40a3-b647-a57c001931d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:57.123805 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82face-d287-40a3-b647-a57c001931d9 admin admin] PATH_INFO: `/v3/projects/f65e2aee89244ff8b1406853f94ffdb9` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:57.125314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82face-d287-40a3-b647-a57c001931d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f65e2aee89244ff8b1406853f94ffdb9)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:57.128978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82face-d287-40a3-b647-a57c001931d9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:57.190359 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:57.208678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9c82face-d287-40a3-b647-a57c001931d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f65e2aee89244ff8b1406853f94ffdb9', 'request_id': 'req-9c82face-d287-40a3-b647-a57c001931d9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:57.216797 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9a02a-a125-499b-98db-360cbe258f03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:57.219394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9a02a-a125-499b-98db-360cbe258f03 service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:57.220173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9c82face-d287-40a3-b647-a57c001931d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f65e2aee89244ff8b1406853f94ffdb9', 'request_id': 'req-9c82face-d287-40a3-b647-a57c001931d9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:57.228414 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9c82face-d287-40a3-b647-a57c001931d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f65e2aee89244ff8b1406853f94ffdb9', 'request_id': 'req-9c82face-d287-40a3-b647-a57c001931d9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:57.231809 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1406/2834] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:10:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:10:57.248248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9c82face-d287-40a3-b647-a57c001931d9 admin admin] The token cache is being invalidate because project f65e2aee89244ff8b1406853f94ffdb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:10:57.248248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9c82face-d287-40a3-b647-a57c001931d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:57.249003 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-9c82face-d287-40a3-b647-a57c001931d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:57.250873 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1429/2835] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:10:57 2022] DELETE /identity/v3/projects/f65e2aee89244ff8b1406853f94ffdb9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:57.256993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af526703-ca12-41b9-968b-61c39c6a53d3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:57.257289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af526703-ca12-41b9-968b-61c39c6a53d3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:57.257634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af526703-ca12-41b9-968b-61c39c6a53d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:57.282604 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-af526703-ca12-41b9-968b-61c39c6a53d3 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:57.283880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-af526703-ca12-41b9-968b-61c39c6a53d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:57.393075 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1407/2836] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:57.400634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0db2e-7fb5-4bdf-93a9-17a8de20b866 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:57.401127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0db2e-7fb5-4bdf-93a9-17a8de20b866 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:57.401215 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0db2e-7fb5-4bdf-93a9-17a8de20b866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:57.438317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-2da0db2e-7fb5-4bdf-93a9-17a8de20b866 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:57.439811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2da0db2e-7fb5-4bdf-93a9-17a8de20b866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:57.453685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1430/2837] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:57.455029 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af526703-ca12-41b9-968b-61c39c6a53d3 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:57.471326 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af526703-ca12-41b9-968b-61c39c6a53d3 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:57.478195 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681993a-8f5e-4466-b1e7-4528730603f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:57.478695 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681993a-8f5e-4466-b1e7-4528730603f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:57.479145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681993a-8f5e-4466-b1e7-4528730603f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:57.479233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681993a-8f5e-4466-b1e7-4528730603f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:57.479871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681993a-8f5e-4466-b1e7-4528730603f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:10:57.480219 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681993a-8f5e-4466-b1e7-4528730603f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:57.481860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681993a-8f5e-4466-b1e7-4528730603f1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:57.497790 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1408/2838] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:10:57.503085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0db2e-7fb5-4bdf-93a9-17a8de20b866 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:57.514248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0db2e-7fb5-4bdf-93a9-17a8de20b866 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:57.522070 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42741069-82d9-43ce-b3a8-d4301770abbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:57.523272 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42741069-82d9-43ce-b3a8-d4301770abbf admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:57.523641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42741069-82d9-43ce-b3a8-d4301770abbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:57.524093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42741069-82d9-43ce-b3a8-d4301770abbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:57.525218 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42741069-82d9-43ce-b3a8-d4301770abbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:57.527705 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42741069-82d9-43ce-b3a8-d4301770abbf admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:57.581437 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1431/2839] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:10:57 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:10:57.581681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681993a-8f5e-4466-b1e7-4528730603f1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:57.602606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681993a-8f5e-4466-b1e7-4528730603f1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:57.696544 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c57da9-9a1f-49e6-ab69-91234d30d913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:57.697267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c57da9-9a1f-49e6-ab69-91234d30d913 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:57.697810 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c57da9-9a1f-49e6-ab69-91234d30d913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:57.698095 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c57da9-9a1f-49e6-ab69-91234d30d913 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:57.699467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c57da9-9a1f-49e6-ab69-91234d30d913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:57.701616 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c57da9-9a1f-49e6-ab69-91234d30d913 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:57.746671 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1409/2840] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:10:57 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:10:57.748154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42741069-82d9-43ce-b3a8-d4301770abbf admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:57.766526 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42741069-82d9-43ce-b3a8-d4301770abbf admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:57.773105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6feea9-6dee-4dd4-a8fb-7b71c73f96bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:57.773736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6feea9-6dee-4dd4-a8fb-7b71c73f96bc admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:57.774002 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6feea9-6dee-4dd4-a8fb-7b71c73f96bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:57.774270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6feea9-6dee-4dd4-a8fb-7b71c73f96bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:57.774917 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6feea9-6dee-4dd4-a8fb-7b71c73f96bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:57.776951 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6feea9-6dee-4dd4-a8fb-7b71c73f96bc admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:57.799756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1432/2841] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:10:57.803598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c57da9-9a1f-49e6-ab69-91234d30d913 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:57.825046 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c57da9-9a1f-49e6-ab69-91234d30d913 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:57.833476 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97debfd-9541-4a9c-8c93-a9313fbb122d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:57.834526 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97debfd-9541-4a9c-8c93-a9313fbb122d admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:57.834698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97debfd-9541-4a9c-8c93-a9313fbb122d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:57.835483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97debfd-9541-4a9c-8c93-a9313fbb122d admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:57.836466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97debfd-9541-4a9c-8c93-a9313fbb122d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:57.838363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97debfd-9541-4a9c-8c93-a9313fbb122d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:57.884061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1410/2842] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:10:57.892027 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6feea9-6dee-4dd4-a8fb-7b71c73f96bc admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:57.902862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6feea9-6dee-4dd4-a8fb-7b71c73f96bc admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:57.977035 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d92970-b610-4318-95d0-5a4c65a4a27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:57.977916 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d92970-b610-4318-95d0-5a4c65a4a27b admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:57.978326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d92970-b610-4318-95d0-5a4c65a4a27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:57.978570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d92970-b610-4318-95d0-5a4c65a4a27b admin admin] PATH_INFO: `/v3/projects/52ebc0bd87d741feb69ee6eb91aa0c8b/users/1f0f0368edca43ebaaefa28cd36bfbf1/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:58.023222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d92970-b610-4318-95d0-5a4c65a4a27b admin admin] RBAC: Authorizing `identity:create_grant(project_id=52ebc0bd87d741feb69ee6eb91aa0c8b, user_id=1f0f0368edca43ebaaefa28cd36bfbf1, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:58.025964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d92970-b610-4318-95d0-5a4c65a4a27b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:58.047441 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1433/2843] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:10:57 2022] PUT /identity/v3/projects/52ebc0bd87d741feb69ee6eb91aa0c8b/users/1f0f0368edca43ebaaefa28cd36bfbf1/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:58.059761 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3255b-dbf8-4846-a768-c12b82214215 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:58.060310 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3255b-dbf8-4846-a768-c12b82214215 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:58.060394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3255b-dbf8-4846-a768-c12b82214215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:58.115848 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8ec3255b-dbf8-4846-a768-c12b82214215 None None] MFA Rules not processed for user `1f0f0368edca43ebaaefa28cd36bfbf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:58.119521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8ec3255b-dbf8-4846-a768-c12b82214215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:58.222679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:58.284987 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1411/2844] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:58.320239 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d92970-b610-4318-95d0-5a4c65a4a27b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:58.322276 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-99d92970-b610-4318-95d0-5a4c65a4a27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:58.347445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:58.459049 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d92970-b610-4318-95d0-5a4c65a4a27b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:58.467077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36196649-2a94-4f96-9306-0cbefdd4dcc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:58.468877 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36196649-2a94-4f96-9306-0cbefdd4dcc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:58.469363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36196649-2a94-4f96-9306-0cbefdd4dcc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:58.469710 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36196649-2a94-4f96-9306-0cbefdd4dcc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:58.493815 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36196649-2a94-4f96-9306-0cbefdd4dcc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:58.495699 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36196649-2a94-4f96-9306-0cbefdd4dcc2 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:58.536273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1434/2845] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:10:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:10:58.564507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3255b-dbf8-4846-a768-c12b82214215 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:58.651287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3255b-dbf8-4846-a768-c12b82214215 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:58.656318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1bb83-f8f8-45ea-b4c0-3e446b8d9df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:58.656959 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1bb83-f8f8-45ea-b4c0-3e446b8d9df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:58.657241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1bb83-f8f8-45ea-b4c0-3e446b8d9df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:58.657515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1bb83-f8f8-45ea-b4c0-3e446b8d9df3 admin admin] PATH_INFO: `/v3/users/1f0f0368edca43ebaaefa28cd36bfbf1` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:58.658696 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1bb83-f8f8-45ea-b4c0-3e446b8d9df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f0f0368edca43ebaaefa28cd36bfbf1)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:58.661232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1bb83-f8f8-45ea-b4c0-3e446b8d9df3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:58.715134 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4dc1bb83-f8f8-45ea-b4c0-3e446b8d9df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f0f0368edca43ebaaefa28cd36bfbf1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:58.725334 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4dc1bb83-f8f8-45ea-b4c0-3e446b8d9df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f0f0368edca43ebaaefa28cd36bfbf1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:58.747177 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4dc1bb83-f8f8-45ea-b4c0-3e446b8d9df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f0f0368edca43ebaaefa28cd36bfbf1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:58.748787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4dc1bb83-f8f8-45ea-b4c0-3e446b8d9df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f0f0368edca43ebaaefa28cd36bfbf1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:58.750054 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4dc1bb83-f8f8-45ea-b4c0-3e446b8d9df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f0f0368edca43ebaaefa28cd36bfbf1'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:58.777444 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1412/2846] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:10:58 2022] DELETE /identity/v3/users/1f0f0368edca43ebaaefa28cd36bfbf1 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:10:58.785828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8added25-e1e1-4aad-adda-f5e8afa0b1bb None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:58.786116 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8added25-e1e1-4aad-adda-f5e8afa0b1bb None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:58.786397 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8added25-e1e1-4aad-adda-f5e8afa0b1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:58.786781 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b7db6-1b8e-4beb-9432-5095ce7c3f08 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:58.787283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b7db6-1b8e-4beb-9432-5095ce7c3f08 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:58.787816 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b7db6-1b8e-4beb-9432-5095ce7c3f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:58.819180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-8added25-e1e1-4aad-adda-f5e8afa0b1bb None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:58.820584 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8added25-e1e1-4aad-adda-f5e8afa0b1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:58.821473 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-0e8b7db6-1b8e-4beb-9432-5095ce7c3f08 None None] MFA Rules not processed for user `309d1efd676845bbaa30f472b19ffa34`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:10:58.823213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0e8b7db6-1b8e-4beb-9432-5095ce7c3f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:58.887460 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1435/2847] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:58.898235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1413/2848] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:10:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:10:58.902507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8added25-e1e1-4aad-adda-f5e8afa0b1bb None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:58.904303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-8added25-e1e1-4aad-adda-f5e8afa0b1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:58.986172 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8added25-e1e1-4aad-adda-f5e8afa0b1bb None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:58.991189 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56804c-a894-4b80-9463-4d92e9aa74ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:58.991824 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56804c-a894-4b80-9463-4d92e9aa74ea service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:58.992146 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56804c-a894-4b80-9463-4d92e9aa74ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:58.992424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56804c-a894-4b80-9463-4d92e9aa74ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:59.015709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56804c-a894-4b80-9463-4d92e9aa74ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:59.017671 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56804c-a894-4b80-9463-4d92e9aa74ea service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:59.040332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1436/2849] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:10:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:10:59.375072 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b7db6-1b8e-4beb-9432-5095ce7c3f08 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:10:59.377859 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-0e8b7db6-1b8e-4beb-9432-5095ce7c3f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:10:59.400036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:10:59.412302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b7db6-1b8e-4beb-9432-5095ce7c3f08 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:10:59.422429 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:10:59.423871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:10:59.424201 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:10:59.425295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] PATH_INFO: `/v3/projects/52ebc0bd87d741feb69ee6eb91aa0c8b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:10:59.428467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52ebc0bd87d741feb69ee6eb91aa0c8b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:10:59.430757 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:10:59.511511 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52ebc0bd87d741feb69ee6eb91aa0c8b', 'request_id': 'req-ff3d7a62-2027-4863-8597-91e75a395e85'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:59.526287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52ebc0bd87d741feb69ee6eb91aa0c8b', 'request_id': 'req-ff3d7a62-2027-4863-8597-91e75a395e85'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:59.534672 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52ebc0bd87d741feb69ee6eb91aa0c8b', 'request_id': 'req-ff3d7a62-2027-4863-8597-91e75a395e85'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:59.546193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] The token cache is being invalidate because project 52ebc0bd87d741feb69ee6eb91aa0c8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:10:59.546476 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:59.547132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:10:59.548844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1414/2850] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:10:59 2022] DELETE /identity/v3/projects/52ebc0bd87d741feb69ee6eb91aa0c8b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:11:09.024423 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=76 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:11:09.543152 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:11:22.650980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56804c-a894-4b80-9463-4d92e9aa74ea service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:11:22.653100 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4c56804c-a894-4b80-9463-4d92e9aa74ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:11:22.705980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56804c-a894-4b80-9463-4d92e9aa74ea service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:11:22.773470 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c3574-0ebd-4439-9d3c-d6b024722d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:11:22.774005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c3574-0ebd-4439-9d3c-d6b024722d2d service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:11:22.774283 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c3574-0ebd-4439-9d3c-d6b024722d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:11:22.774586 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c3574-0ebd-4439-9d3c-d6b024722d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:11:22.776322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d35c3574-0ebd-4439-9d3c-d6b024722d2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:11:22.786268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c3574-0ebd-4439-9d3c-d6b024722d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:11:22.787936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c3574-0ebd-4439-9d3c-d6b024722d2d service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:11:22.800154 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1437/2851] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:11:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:11:32.796689 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=39 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:11:38.680877 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:11:38.682311 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:11:38.697413 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d7a62-2027-4863-8597-91e75a395e85 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:11:38.701189 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae56221-08e1-4e52-91b3-ebde6ad0ddd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:11:38.701858 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae56221-08e1-4e52-91b3-ebde6ad0ddd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:11:38.702131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae56221-08e1-4e52-91b3-ebde6ad0ddd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:11:38.702398 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae56221-08e1-4e52-91b3-ebde6ad0ddd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:11:38.704084 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4ae56221-08e1-4e52-91b3-ebde6ad0ddd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:11:38.766914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae56221-08e1-4e52-91b3-ebde6ad0ddd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:11:38.768268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae56221-08e1-4e52-91b3-ebde6ad0ddd4 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:11:38.778000 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1415/2852] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:11:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:11:47.765031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c3574-0ebd-4439-9d3c-d6b024722d2d service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:11:47.766493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d35c3574-0ebd-4439-9d3c-d6b024722d2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:11:47.826232 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c3574-0ebd-4439-9d3c-d6b024722d2d service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:11:47.829988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172fe2-36de-4104-92cd-2e8207da0861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:11:47.830595 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172fe2-36de-4104-92cd-2e8207da0861 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:11:47.830877 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172fe2-36de-4104-92cd-2e8207da0861 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:11:47.831153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172fe2-36de-4104-92cd-2e8207da0861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:11:47.832984 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4e172fe2-36de-4104-92cd-2e8207da0861 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:11:47.904790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e172fe2-36de-4104-92cd-2e8207da0861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:11:47.906233 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e172fe2-36de-4104-92cd-2e8207da0861 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:11:47.925144 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1438/2853] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:11:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:11:48.773709 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=32 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:11:57.922113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=56 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:12:48.585591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae56221-08e1-4e52-91b3-ebde6ad0ddd4 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:12:48.587202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4ae56221-08e1-4e52-91b3-ebde6ad0ddd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:12:48.599564 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae56221-08e1-4e52-91b3-ebde6ad0ddd4 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:12:48.603546 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5202fa-d6c4-40a7-8fa1-c03b2496f014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:12:48.604209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5202fa-d6c4-40a7-8fa1-c03b2496f014 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:48.604491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5202fa-d6c4-40a7-8fa1-c03b2496f014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:48.604762 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5202fa-d6c4-40a7-8fa1-c03b2496f014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:48.606550 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-bd5202fa-d6c4-40a7-8fa1-c03b2496f014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:12:48.688540 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5202fa-d6c4-40a7-8fa1-c03b2496f014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:12:48.689915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5202fa-d6c4-40a7-8fa1-c03b2496f014 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:12:48.711118 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1416/2854] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:12:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:12:50.239854 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172fe2-36de-4104-92cd-2e8207da0861 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:12:50.242844 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4e172fe2-36de-4104-92cd-2e8207da0861 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:12:50.262284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172fe2-36de-4104-92cd-2e8207da0861 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:12:50.268996 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123993b7-469f-4583-8818-23f276c793cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:12:50.269621 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123993b7-469f-4583-8818-23f276c793cd service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:50.270048 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123993b7-469f-4583-8818-23f276c793cd service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:50.270326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123993b7-469f-4583-8818-23f276c793cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:50.274342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-123993b7-469f-4583-8818-23f276c793cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:12:50.290554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123993b7-469f-4583-8818-23f276c793cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:12:50.292559 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123993b7-469f-4583-8818-23f276c793cd service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:12:50.324429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1439/2855] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:12:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:12:54.751346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5202fa-d6c4-40a7-8fa1-c03b2496f014 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:12:54.760358 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5202fa-d6c4-40a7-8fa1-c03b2496f014 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:12:54.764760 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89a5d9-8313-4679-8cea-2c14697ed20a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:12:54.765366 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89a5d9-8313-4679-8cea-2c14697ed20a service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:54.765650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89a5d9-8313-4679-8cea-2c14697ed20a service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:54.765921 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89a5d9-8313-4679-8cea-2c14697ed20a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:54.767756 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4f89a5d9-8313-4679-8cea-2c14697ed20a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:12:54.836309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89a5d9-8313-4679-8cea-2c14697ed20a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:12:54.837808 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89a5d9-8313-4679-8cea-2c14697ed20a service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:12:54.848318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1417/2856] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:12:58.093261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123993b7-469f-4583-8818-23f276c793cd service nova] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:12:58.095076 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-123993b7-469f-4583-8818-23f276c793cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:12:58.126990 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123993b7-469f-4583-8818-23f276c793cd service nova] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:12:58.132601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494abda7-848e-4b5c-877c-03293c184e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:12:58.133371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494abda7-848e-4b5c-877c-03293c184e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:58.133747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494abda7-848e-4b5c-877c-03293c184e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:58.134126 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494abda7-848e-4b5c-877c-03293c184e40 admin admin] PATH_INFO: `/v3/users/f4c978e96d3248d59120c287ecc75e87` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:58.135543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494abda7-848e-4b5c-877c-03293c184e40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4c978e96d3248d59120c287ecc75e87)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:12:58.137660 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494abda7-848e-4b5c-877c-03293c184e40 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:12:58.198685 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-494abda7-848e-4b5c-877c-03293c184e40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4c978e96d3248d59120c287ecc75e87'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:12:58.199449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-494abda7-848e-4b5c-877c-03293c184e40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4c978e96d3248d59120c287ecc75e87'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:12:58.206965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-494abda7-848e-4b5c-877c-03293c184e40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4c978e96d3248d59120c287ecc75e87'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:12:58.214420 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-494abda7-848e-4b5c-877c-03293c184e40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4c978e96d3248d59120c287ecc75e87'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:12:58.227872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-494abda7-848e-4b5c-877c-03293c184e40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4c978e96d3248d59120c287ecc75e87'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:12:58.234490 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1440/2857] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:12:58 2022] DELETE /identity/v3/users/f4c978e96d3248d59120c287ecc75e87 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:12:58.415712 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89a5d9-8313-4679-8cea-2c14697ed20a service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:12:58.436260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89a5d9-8313-4679-8cea-2c14697ed20a service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:12:58.502686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:12:58.503224 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:58.503501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:58.503767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0 admin admin] PATH_INFO: `/v3/projects/60788ee7f0ab439087f63db271b65ef4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:58.504880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60788ee7f0ab439087f63db271b65ef4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:12:58.506767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:12:58.512844 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:12:58.553568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60788ee7f0ab439087f63db271b65ef4', 'request_id': 'req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:12:58.560905 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60788ee7f0ab439087f63db271b65ef4', 'request_id': 'req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:12:58.567282 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60788ee7f0ab439087f63db271b65ef4', 'request_id': 'req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:12:58.578581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0 admin admin] The token cache is being invalidate because project 60788ee7f0ab439087f63db271b65ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:12:58.578913 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:12:58.579587 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-9fe00b05-f1f1-4545-b3f3-f5060a9cc8b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:12:58.581164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1418/2858] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:12:58 2022] DELETE /identity/v3/projects/60788ee7f0ab439087f63db271b65ef4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:12:58.587173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959761ce-d438-4699-a658-7f338d4f5855 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:58.587459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959761ce-d438-4699-a658-7f338d4f5855 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:58.587735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959761ce-d438-4699-a658-7f338d4f5855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:58.607936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-959761ce-d438-4699-a658-7f338d4f5855 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:12:58.609178 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-959761ce-d438-4699-a658-7f338d4f5855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:12:58.669180 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1441/2859] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:12:58.677209 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae3565-0bd9-4dbd-bca0-30e6ae7a2eae None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:58.677496 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae3565-0bd9-4dbd-bca0-30e6ae7a2eae None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:58.677768 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae3565-0bd9-4dbd-bca0-30e6ae7a2eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:58.698979 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-1cae3565-0bd9-4dbd-bca0-30e6ae7a2eae None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:12:58.700234 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1cae3565-0bd9-4dbd-bca0-30e6ae7a2eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:12:58.712525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1419/2860] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:12:58.714024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959761ce-d438-4699-a658-7f338d4f5855 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:12:58.724944 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959761ce-d438-4699-a658-7f338d4f5855 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:12:58.729594 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665d9b1-2203-475f-bb4d-4ee2c8f89acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:12:58.730201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665d9b1-2203-475f-bb4d-4ee2c8f89acd admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:58.730480 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665d9b1-2203-475f-bb4d-4ee2c8f89acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:58.730751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665d9b1-2203-475f-bb4d-4ee2c8f89acd admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:58.731395 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665d9b1-2203-475f-bb4d-4ee2c8f89acd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:12:58.731725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665d9b1-2203-475f-bb4d-4ee2c8f89acd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:12:58.733604 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665d9b1-2203-475f-bb4d-4ee2c8f89acd admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:12:58.745138 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1442/2861] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:12:58.750071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae3565-0bd9-4dbd-bca0-30e6ae7a2eae None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:12:58.759322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae3565-0bd9-4dbd-bca0-30e6ae7a2eae None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:12:58.765206 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659d464-4d5a-40d8-8524-d0706cfe664a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:12:58.766264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659d464-4d5a-40d8-8524-d0706cfe664a admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:58.766615 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659d464-4d5a-40d8-8524-d0706cfe664a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:58.766890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659d464-4d5a-40d8-8524-d0706cfe664a admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:58.767519 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659d464-4d5a-40d8-8524-d0706cfe664a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:12:58.769439 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659d464-4d5a-40d8-8524-d0706cfe664a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:12:58.803144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1420/2862] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:12:58 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:12:58.807204 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665d9b1-2203-475f-bb4d-4ee2c8f89acd admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:12:58.817912 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665d9b1-2203-475f-bb4d-4ee2c8f89acd admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:12:58.861169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:12:58.876756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b56b63-ce41-408b-b1ad-830a3ac12945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:12:58.877361 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b56b63-ce41-408b-b1ad-830a3ac12945 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:58.877640 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b56b63-ce41-408b-b1ad-830a3ac12945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:58.877935 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b56b63-ce41-408b-b1ad-830a3ac12945 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:58.878554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b56b63-ce41-408b-b1ad-830a3ac12945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:12:58.880484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b56b63-ce41-408b-b1ad-830a3ac12945 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:12:58.901600 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1443/2863] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:12:58 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:12:58.906397 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659d464-4d5a-40d8-8524-d0706cfe664a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:12:58.916970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659d464-4d5a-40d8-8524-d0706cfe664a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:12:58.922582 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9624f-0fd0-4e63-be34-193fdeab168c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:12:58.923173 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9624f-0fd0-4e63-be34-193fdeab168c admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:58.923448 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9624f-0fd0-4e63-be34-193fdeab168c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:58.923716 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9624f-0fd0-4e63-be34-193fdeab168c admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:58.924378 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9624f-0fd0-4e63-be34-193fdeab168c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:12:58.926050 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9624f-0fd0-4e63-be34-193fdeab168c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:12:58.936507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1421/2864] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:12:58.940721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b56b63-ce41-408b-b1ad-830a3ac12945 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:12:58.950557 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b56b63-ce41-408b-b1ad-830a3ac12945 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:12:58.955401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563f5d7-5379-4191-bc2d-2bd92487dcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:12:58.956009 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563f5d7-5379-4191-bc2d-2bd92487dcb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:58.956303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563f5d7-5379-4191-bc2d-2bd92487dcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:58.956574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563f5d7-5379-4191-bc2d-2bd92487dcb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:58.957203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563f5d7-5379-4191-bc2d-2bd92487dcb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:12:58.958778 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563f5d7-5379-4191-bc2d-2bd92487dcb5 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:12:58.969962 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1444/2865] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:12:58.974577 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9624f-0fd0-4e63-be34-193fdeab168c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:12:58.985510 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9624f-0fd0-4e63-be34-193fdeab168c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:12:58.990289 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a33527-0ddd-4f6e-9d85-165a8620ae86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:12:58.991407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a33527-0ddd-4f6e-9d85-165a8620ae86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:58.991538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a33527-0ddd-4f6e-9d85-165a8620ae86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:58.991875 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a33527-0ddd-4f6e-9d85-165a8620ae86 admin admin] PATH_INFO: `/v3/projects/8012dc5fde7946fd8e0315757e7c83bb/users/036a3b5a91854a7885a36c160356c884/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:59.007210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a33527-0ddd-4f6e-9d85-165a8620ae86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8012dc5fde7946fd8e0315757e7c83bb, user_id=036a3b5a91854a7885a36c160356c884, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:12:59.009936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a33527-0ddd-4f6e-9d85-165a8620ae86 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:12:59.023454 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1422/2866] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:12:58 2022] PUT /identity/v3/projects/8012dc5fde7946fd8e0315757e7c83bb/users/036a3b5a91854a7885a36c160356c884/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:12:59.034626 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de108-0106-463f-81ff-85f59136f917 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:59.034911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de108-0106-463f-81ff-85f59136f917 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:59.035181 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67de108-0106-463f-81ff-85f59136f917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:59.064190 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-c67de108-0106-463f-81ff-85f59136f917 None None] MFA Rules not processed for user `036a3b5a91854a7885a36c160356c884`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:12:59.065451 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c67de108-0106-463f-81ff-85f59136f917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:12:59.128484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1445/2867] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:12:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:12:59.140669 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a33527-0ddd-4f6e-9d85-165a8620ae86 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:12:59.142339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b5a33527-0ddd-4f6e-9d85-165a8620ae86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:12:59.217296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a33527-0ddd-4f6e-9d85-165a8620ae86 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:12:59.222778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c528b-e6b6-4d82-b1d9-3e19f82f4abd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:12:59.223379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c528b-e6b6-4d82-b1d9-3e19f82f4abd service nova] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:12:59.223662 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c528b-e6b6-4d82-b1d9-3e19f82f4abd service nova] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:12:59.223937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c528b-e6b6-4d82-b1d9-3e19f82f4abd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:12:59.231646 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c528b-e6b6-4d82-b1d9-3e19f82f4abd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:12:59.233802 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c528b-e6b6-4d82-b1d9-3e19f82f4abd service nova] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:12:59.242595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1423/2868] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:12:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:13:08.831185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67de108-0106-463f-81ff-85f59136f917 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:08.895488 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67de108-0106-463f-81ff-85f59136f917 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:08.900273 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:08.900897 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:08.901173 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:08.901441 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] PATH_INFO: `/v3/users/036a3b5a91854a7885a36c160356c884` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:08.902551 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=036a3b5a91854a7885a36c160356c884)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:08.905080 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:08.952677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '036a3b5a91854a7885a36c160356c884'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:08.953339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '036a3b5a91854a7885a36c160356c884'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:08.960516 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '036a3b5a91854a7885a36c160356c884'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:08.962191 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:13:08.969236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '036a3b5a91854a7885a36c160356c884'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:08.983704 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '036a3b5a91854a7885a36c160356c884'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:08.990193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1446/2869] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:13:08 2022] DELETE /identity/v3/users/036a3b5a91854a7885a36c160356c884 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:13:08.994694 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dca7b3-b582-4bce-bfff-8842c55e90f4 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:08.995194 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dca7b3-b582-4bce-bfff-8842c55e90f4 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:08.995290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dca7b3-b582-4bce-bfff-8842c55e90f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:09.016273 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-69dca7b3-b582-4bce-bfff-8842c55e90f4 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:13:09.017487 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-69dca7b3-b582-4bce-bfff-8842c55e90f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:13:09.027666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:13:09.086411 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1424/2870] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:13:09.096227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:09.097628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:13:09.156065 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625e0bf-8bfc-4866-875e-2f4e002e1972 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:09.159857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d69250-07a7-4a5d-8145-883eef8b2dc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:09.160519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d69250-07a7-4a5d-8145-883eef8b2dc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:09.160923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d69250-07a7-4a5d-8145-883eef8b2dc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:09.161201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d69250-07a7-4a5d-8145-883eef8b2dc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:09.171088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d69250-07a7-4a5d-8145-883eef8b2dc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:09.172554 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d69250-07a7-4a5d-8145-883eef8b2dc1 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:09.182415 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1447/2871] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:13:09.701401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dca7b3-b582-4bce-bfff-8842c55e90f4 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:09.703600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-69dca7b3-b582-4bce-bfff-8842c55e90f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:13:09.719006 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dca7b3-b582-4bce-bfff-8842c55e90f4 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:09.724024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07068a6a-8e17-4f23-aa20-abcca51357c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:09.724687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07068a6a-8e17-4f23-aa20-abcca51357c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:09.724972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07068a6a-8e17-4f23-aa20-abcca51357c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:09.725249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07068a6a-8e17-4f23-aa20-abcca51357c6 admin admin] PATH_INFO: `/v3/projects/8012dc5fde7946fd8e0315757e7c83bb` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:09.726409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07068a6a-8e17-4f23-aa20-abcca51357c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8012dc5fde7946fd8e0315757e7c83bb)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:09.728432 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07068a6a-8e17-4f23-aa20-abcca51357c6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:09.782944 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-07068a6a-8e17-4f23-aa20-abcca51357c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8012dc5fde7946fd8e0315757e7c83bb', 'request_id': 'req-07068a6a-8e17-4f23-aa20-abcca51357c6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:09.790137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-07068a6a-8e17-4f23-aa20-abcca51357c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8012dc5fde7946fd8e0315757e7c83bb', 'request_id': 'req-07068a6a-8e17-4f23-aa20-abcca51357c6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:09.800149 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-07068a6a-8e17-4f23-aa20-abcca51357c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8012dc5fde7946fd8e0315757e7c83bb', 'request_id': 'req-07068a6a-8e17-4f23-aa20-abcca51357c6'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:09.815048 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-07068a6a-8e17-4f23-aa20-abcca51357c6 admin admin] The token cache is being invalidate because project 8012dc5fde7946fd8e0315757e7c83bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:13:09.815347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-07068a6a-8e17-4f23-aa20-abcca51357c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:09.816070 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-07068a6a-8e17-4f23-aa20-abcca51357c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:09.817752 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1425/2872] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:13:09 2022] DELETE /identity/v3/projects/8012dc5fde7946fd8e0315757e7c83bb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:13:09.822521 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2cbc0-e72e-4c59-b9ff-77fb970ca3a4 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:09.822811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2cbc0-e72e-4c59-b9ff-77fb970ca3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:09.823084 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2cbc0-e72e-4c59-b9ff-77fb970ca3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:09.842771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a4f2cbc0-e72e-4c59-b9ff-77fb970ca3a4 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:13:09.843977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a4f2cbc0-e72e-4c59-b9ff-77fb970ca3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:13:09.906162 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1448/2873] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:13:09.915056 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e8721-58ab-4c9a-80b6-4195dc07bbd8 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:09.915344 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e8721-58ab-4c9a-80b6-4195dc07bbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:09.915622 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e8721-58ab-4c9a-80b6-4195dc07bbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:09.937099 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-2b7e8721-58ab-4c9a-80b6-4195dc07bbd8 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:13:09.938309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2b7e8721-58ab-4c9a-80b6-4195dc07bbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:13:09.946453 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1426/2874] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:13:09.950677 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2cbc0-e72e-4c59-b9ff-77fb970ca3a4 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:09.960977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2cbc0-e72e-4c59-b9ff-77fb970ca3a4 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:09.965536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15b5fa-7780-4320-b9cc-38ee26ad2d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:09.966828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15b5fa-7780-4320-b9cc-38ee26ad2d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:09.967167 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15b5fa-7780-4320-b9cc-38ee26ad2d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:09.967448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15b5fa-7780-4320-b9cc-38ee26ad2d4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:09.968134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15b5fa-7780-4320-b9cc-38ee26ad2d4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:13:09.968443 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15b5fa-7780-4320-b9cc-38ee26ad2d4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:09.970039 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15b5fa-7780-4320-b9cc-38ee26ad2d4a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:09.984032 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1449/2875] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:13:09.988685 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e8721-58ab-4c9a-80b6-4195dc07bbd8 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:10.001249 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e8721-58ab-4c9a-80b6-4195dc07bbd8 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:10.006030 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65211510-29a1-4ec7-b5e5-aea309964f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:10.006633 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65211510-29a1-4ec7-b5e5-aea309964f09 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:10.006917 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65211510-29a1-4ec7-b5e5-aea309964f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:10.007323 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65211510-29a1-4ec7-b5e5-aea309964f09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:10.012379 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65211510-29a1-4ec7-b5e5-aea309964f09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:10.014401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65211510-29a1-4ec7-b5e5-aea309964f09 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:10.043321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1427/2876] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:13:09 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:13:10.048014 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15b5fa-7780-4320-b9cc-38ee26ad2d4a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:10.058194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15b5fa-7780-4320-b9cc-38ee26ad2d4a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:10.116665 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9138999-870b-4e3d-aee3-3b6dca1a5fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:10.117281 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9138999-870b-4e3d-aee3-3b6dca1a5fab admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:10.117683 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9138999-870b-4e3d-aee3-3b6dca1a5fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:10.117960 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9138999-870b-4e3d-aee3-3b6dca1a5fab admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:10.118597 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9138999-870b-4e3d-aee3-3b6dca1a5fab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:10.120535 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9138999-870b-4e3d-aee3-3b6dca1a5fab admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:10.130966 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:13:10.144536 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1450/2877] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:13:10 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:13:10.148959 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65211510-29a1-4ec7-b5e5-aea309964f09 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:10.159988 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65211510-29a1-4ec7-b5e5-aea309964f09 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:10.164751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82064604-7007-4af3-9a6b-2b10dda20281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:10.165363 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82064604-7007-4af3-9a6b-2b10dda20281 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:10.165647 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82064604-7007-4af3-9a6b-2b10dda20281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:10.165920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82064604-7007-4af3-9a6b-2b10dda20281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:10.166566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82064604-7007-4af3-9a6b-2b10dda20281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:10.168210 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82064604-7007-4af3-9a6b-2b10dda20281 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:10.179421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1428/2878] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:13:10.183627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9138999-870b-4e3d-aee3-3b6dca1a5fab admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:10.193950 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9138999-870b-4e3d-aee3-3b6dca1a5fab admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:10.198648 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbf87b-b101-4d37-a366-a9534c12173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:10.199194 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbf87b-b101-4d37-a366-a9534c12173a admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:10.199477 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbf87b-b101-4d37-a366-a9534c12173a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:10.199747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbf87b-b101-4d37-a366-a9534c12173a admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:10.200428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbf87b-b101-4d37-a366-a9534c12173a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:10.202151 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbf87b-b101-4d37-a366-a9534c12173a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:10.212872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1451/2879] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:13:10.217592 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82064604-7007-4af3-9a6b-2b10dda20281 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:10.227690 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82064604-7007-4af3-9a6b-2b10dda20281 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:10.233598 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f559477-9288-4916-bba2-e6c1e8779fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:10.234256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f559477-9288-4916-bba2-e6c1e8779fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:10.234606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f559477-9288-4916-bba2-e6c1e8779fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:10.234835 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f559477-9288-4916-bba2-e6c1e8779fbe admin admin] PATH_INFO: `/v3/projects/d1326920f54545858c4e96c67efae522/users/4960183187d04549997bfbb2deb3032f/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:10.250718 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f559477-9288-4916-bba2-e6c1e8779fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1326920f54545858c4e96c67efae522, user_id=4960183187d04549997bfbb2deb3032f, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:10.253441 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f559477-9288-4916-bba2-e6c1e8779fbe admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:10.266876 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1429/2880] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:13:10 2022] PUT /identity/v3/projects/d1326920f54545858c4e96c67efae522/users/4960183187d04549997bfbb2deb3032f/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:13:10.277068 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f41b32-97ec-47e3-acef-8a97c3f4da47 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:10.277384 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f41b32-97ec-47e3-acef-8a97c3f4da47 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:10.277661 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f41b32-97ec-47e3-acef-8a97c3f4da47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:10.306496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-72f41b32-97ec-47e3-acef-8a97c3f4da47 None None] MFA Rules not processed for user `4960183187d04549997bfbb2deb3032f`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:13:10.307735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-72f41b32-97ec-47e3-acef-8a97c3f4da47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:13:10.376673 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1452/2881] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:13:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:13:10.381171 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f559477-9288-4916-bba2-e6c1e8779fbe admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:10.422501 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:13:10.449911 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f559477-9288-4916-bba2-e6c1e8779fbe admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:10.454796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa18de-5b9e-4c60-8b5c-a237d0915527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:10.455403 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa18de-5b9e-4c60-8b5c-a237d0915527 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:10.455683 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa18de-5b9e-4c60-8b5c-a237d0915527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:10.455957 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa18de-5b9e-4c60-8b5c-a237d0915527 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:10.456724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa18de-5b9e-4c60-8b5c-a237d0915527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:10.458656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa18de-5b9e-4c60-8b5c-a237d0915527 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:10.485968 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1430/2882] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:13:10 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:13:10.490129 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f41b32-97ec-47e3-acef-8a97c3f4da47 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:10.501158 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f41b32-97ec-47e3-acef-8a97c3f4da47 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:10.560200 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6336-c889-4357-805d-c3edc99db3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:10.560809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6336-c889-4357-805d-c3edc99db3ec admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:10.561085 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6336-c889-4357-805d-c3edc99db3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:10.561372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6336-c889-4357-805d-c3edc99db3ec admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:10.562006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b6336-c889-4357-805d-c3edc99db3ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:10.563904 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b6336-c889-4357-805d-c3edc99db3ec admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:10.586295 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1453/2883] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:13:10 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:13:10.590321 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa18de-5b9e-4c60-8b5c-a237d0915527 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:10.601652 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa18de-5b9e-4c60-8b5c-a237d0915527 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:10.612024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db6b79-aa4c-4bf0-9e38-edf890c2ea60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:10.612024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db6b79-aa4c-4bf0-9e38-edf890c2ea60 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:10.612024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db6b79-aa4c-4bf0-9e38-edf890c2ea60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:10.612024 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db6b79-aa4c-4bf0-9e38-edf890c2ea60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:10.613028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db6b79-aa4c-4bf0-9e38-edf890c2ea60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:10.614511 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db6b79-aa4c-4bf0-9e38-edf890c2ea60 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:10.627568 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1431/2884] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:13:10.631921 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6336-c889-4357-805d-c3edc99db3ec admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:10.643374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6336-c889-4357-805d-c3edc99db3ec admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:10.648112 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae9fc2-75eb-4498-9bc2-ac318e8c6a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:10.649051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae9fc2-75eb-4498-9bc2-ac318e8c6a67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:10.649312 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae9fc2-75eb-4498-9bc2-ac318e8c6a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:10.649582 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae9fc2-75eb-4498-9bc2-ac318e8c6a67 admin admin] PATH_INFO: `/v3/projects/b7494caea46f45e691c06f5f29dc15d4/users/530cd06fd76f47a1a0c129e1b4f59733/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:10.664894 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae9fc2-75eb-4498-9bc2-ac318e8c6a67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7494caea46f45e691c06f5f29dc15d4, user_id=530cd06fd76f47a1a0c129e1b4f59733, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:10.667634 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae9fc2-75eb-4498-9bc2-ac318e8c6a67 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:10.681668 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1454/2885] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:13:10 2022] PUT /identity/v3/projects/b7494caea46f45e691c06f5f29dc15d4/users/530cd06fd76f47a1a0c129e1b4f59733/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:13:10.686483 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db6b79-aa4c-4bf0-9e38-edf890c2ea60 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:10.698474 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db6b79-aa4c-4bf0-9e38-edf890c2ea60 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:10.758502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1022e2-97b6-4f19-9afe-9270f03fd4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:10.759023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1022e2-97b6-4f19-9afe-9270f03fd4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:10.759298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1022e2-97b6-4f19-9afe-9270f03fd4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:10.759565 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1022e2-97b6-4f19-9afe-9270f03fd4c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:10.760234 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1022e2-97b6-4f19-9afe-9270f03fd4c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:10.761832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1022e2-97b6-4f19-9afe-9270f03fd4c4 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:10.772985 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1432/2886] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:13:10.777641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae9fc2-75eb-4498-9bc2-ac318e8c6a67 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:10.788019 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae9fc2-75eb-4498-9bc2-ac318e8c6a67 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:10.795927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c8814-386c-4514-aa97-421ebb79dc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:10.796615 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c8814-386c-4514-aa97-421ebb79dc8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:10.796893 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c8814-386c-4514-aa97-421ebb79dc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:10.797159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c8814-386c-4514-aa97-421ebb79dc8b admin admin] PATH_INFO: `/v3/projects/b7494caea46f45e691c06f5f29dc15d4/users/530cd06fd76f47a1a0c129e1b4f59733/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:10.799044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c8814-386c-4514-aa97-421ebb79dc8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7494caea46f45e691c06f5f29dc15d4, user_id=530cd06fd76f47a1a0c129e1b4f59733, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:10.803626 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c8814-386c-4514-aa97-421ebb79dc8b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:10.813277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1455/2887] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:13:10 2022] PUT /identity/v3/projects/b7494caea46f45e691c06f5f29dc15d4/users/530cd06fd76f47a1a0c129e1b4f59733/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:13:10.823508 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55049c29-026d-450d-a4c1-fda45c607398 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:10.823788 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55049c29-026d-450d-a4c1-fda45c607398 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:10.824094 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55049c29-026d-450d-a4c1-fda45c607398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:10.853207 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-55049c29-026d-450d-a4c1-fda45c607398 None None] MFA Rules not processed for user `530cd06fd76f47a1a0c129e1b4f59733`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:13:10.854434 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-55049c29-026d-450d-a4c1-fda45c607398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:13:10.924169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1433/2888] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:13:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:13:10.937355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c8814-386c-4514-aa97-421ebb79dc8b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:10.939140 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2c7c8814-386c-4514-aa97-421ebb79dc8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:13:10.969512 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:13:11.020578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c8814-386c-4514-aa97-421ebb79dc8b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:11.024663 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4330012-324f-46e8-9664-8fc945be479c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:11.025288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4330012-324f-46e8-9664-8fc945be479c service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:11.025565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4330012-324f-46e8-9664-8fc945be479c service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:11.025867 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4330012-324f-46e8-9664-8fc945be479c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:11.036249 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4330012-324f-46e8-9664-8fc945be479c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:11.037692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4330012-324f-46e8-9664-8fc945be479c service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:11.048463 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1456/2889] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:13:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:13:11.127705 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55049c29-026d-450d-a4c1-fda45c607398 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:11.138522 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55049c29-026d-450d-a4c1-fda45c607398 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:11.142145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bdcd9-cc91-42a4-9ddf-af5d67061665 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:11.142822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bdcd9-cc91-42a4-9ddf-af5d67061665 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:11.143168 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bdcd9-cc91-42a4-9ddf-af5d67061665 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:11.143507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bdcd9-cc91-42a4-9ddf-af5d67061665 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:11.174304 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:13:11.211656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bdcd9-cc91-42a4-9ddf-af5d67061665 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:11.213184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bdcd9-cc91-42a4-9ddf-af5d67061665 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:11.222617 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1434/2890] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:13:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:13:21.043071 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=20 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:13:21.218228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=14 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:13:59.361657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4330012-324f-46e8-9664-8fc945be479c service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:59.424090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4330012-324f-46e8-9664-8fc945be479c service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:59.428522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:59.429144 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:59.429424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:59.429697 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] PATH_INFO: `/v3/users/4960183187d04549997bfbb2deb3032f` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:59.430946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4960183187d04549997bfbb2deb3032f)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:59.432932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:59.479003 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4960183187d04549997bfbb2deb3032f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:59.479646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4960183187d04549997bfbb2deb3032f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:59.486471 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4960183187d04549997bfbb2deb3032f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:59.493449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4960183187d04549997bfbb2deb3032f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:59.508203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4960183187d04549997bfbb2deb3032f'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:13:59.515781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1457/2891] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:13:59 2022] DELETE /identity/v3/users/4960183187d04549997bfbb2deb3032f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:13:59.520347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154b1da-eae8-4896-af05-6575a73ec579 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:59.520896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154b1da-eae8-4896-af05-6575a73ec579 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:59.520896 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154b1da-eae8-4896-af05-6575a73ec579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:59.540521 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7154b1da-eae8-4896-af05-6575a73ec579 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:13:59.541724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7154b1da-eae8-4896-af05-6575a73ec579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:13:59.601678 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1435/2892] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:13:59.611519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:13:59.613077 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:13:59.681201 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61346c27-0b71-48a0-aa2d-627dc3bd6677 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:13:59.685016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4b71d-c0cf-477f-b9d2-900a098fb2d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:13:59.685700 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4b71d-c0cf-477f-b9d2-900a098fb2d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:13:59.686122 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4b71d-c0cf-477f-b9d2-900a098fb2d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:13:59.686399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4b71d-c0cf-477f-b9d2-900a098fb2d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:13:59.697196 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4b71d-c0cf-477f-b9d2-900a098fb2d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:13:59.698638 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4b71d-c0cf-477f-b9d2-900a098fb2d8 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:13:59.709751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1458/2893] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:00.005057 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154b1da-eae8-4896-af05-6575a73ec579 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:00.007132 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7154b1da-eae8-4896-af05-6575a73ec579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:00.024871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154b1da-eae8-4896-af05-6575a73ec579 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:00.030591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:00.031415 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:00.031866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:00.032400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] PATH_INFO: `/v3/projects/d1326920f54545858c4e96c67efae522` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:00.033770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1326920f54545858c4e96c67efae522)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:00.035860 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:00.091491 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1326920f54545858c4e96c67efae522', 'request_id': 'req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.101838 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1326920f54545858c4e96c67efae522', 'request_id': 'req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.110704 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1326920f54545858c4e96c67efae522', 'request_id': 'req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.122770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] The token cache is being invalidate because project d1326920f54545858c4e96c67efae522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:14:00.123894 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.124711 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.126553 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1436/2894] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:14:00 2022] DELETE /identity/v3/projects/d1326920f54545858c4e96c67efae522 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:00.130507 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4b71d-c0cf-477f-b9d2-900a098fb2d8 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:00.142105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4b71d-c0cf-477f-b9d2-900a098fb2d8 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:00.164216 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:00.211622 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fbb47-d6f5-4fc7-85df-b98909f185e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:00.212363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fbb47-d6f5-4fc7-85df-b98909f185e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:00.212645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fbb47-d6f5-4fc7-85df-b98909f185e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:00.212920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fbb47-d6f5-4fc7-85df-b98909f185e2 admin admin] PATH_INFO: `/v3/users/530cd06fd76f47a1a0c129e1b4f59733` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:00.214044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936fbb47-d6f5-4fc7-85df-b98909f185e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=530cd06fd76f47a1a0c129e1b4f59733)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:00.215957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936fbb47-d6f5-4fc7-85df-b98909f185e2 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:00.259618 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-936fbb47-d6f5-4fc7-85df-b98909f185e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '530cd06fd76f47a1a0c129e1b4f59733'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.260387 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-936fbb47-d6f5-4fc7-85df-b98909f185e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '530cd06fd76f47a1a0c129e1b4f59733'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.267940 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-936fbb47-d6f5-4fc7-85df-b98909f185e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '530cd06fd76f47a1a0c129e1b4f59733'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.276635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-936fbb47-d6f5-4fc7-85df-b98909f185e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '530cd06fd76f47a1a0c129e1b4f59733'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.288920 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-936fbb47-d6f5-4fc7-85df-b98909f185e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '530cd06fd76f47a1a0c129e1b4f59733'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.295078 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1459/2895] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:14:00 2022] DELETE /identity/v3/users/530cd06fd76f47a1a0c129e1b4f59733 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:00.563945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:00.565163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:00.580293 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b739b8-1cef-4b3c-8eb8-07fda1c83c2c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:00.653279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d13240-1af0-4429-abfc-165a45aefb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:00.654017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d13240-1af0-4429-abfc-165a45aefb07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:00.654981 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d13240-1af0-4429-abfc-165a45aefb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:00.655273 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d13240-1af0-4429-abfc-165a45aefb07 admin admin] PATH_INFO: `/v3/projects/b7494caea46f45e691c06f5f29dc15d4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:00.656410 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d13240-1af0-4429-abfc-165a45aefb07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7494caea46f45e691c06f5f29dc15d4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:00.658335 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d13240-1af0-4429-abfc-165a45aefb07 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:00.689557 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:00.711163 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2d13240-1af0-4429-abfc-165a45aefb07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7494caea46f45e691c06f5f29dc15d4', 'request_id': 'req-e2d13240-1af0-4429-abfc-165a45aefb07'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.720137 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2d13240-1af0-4429-abfc-165a45aefb07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7494caea46f45e691c06f5f29dc15d4', 'request_id': 'req-e2d13240-1af0-4429-abfc-165a45aefb07'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.726794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2d13240-1af0-4429-abfc-165a45aefb07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7494caea46f45e691c06f5f29dc15d4', 'request_id': 'req-e2d13240-1af0-4429-abfc-165a45aefb07'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.737972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2d13240-1af0-4429-abfc-165a45aefb07 admin admin] The token cache is being invalidate because project b7494caea46f45e691c06f5f29dc15d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:14:00.738192 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2d13240-1af0-4429-abfc-165a45aefb07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.738843 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-e2d13240-1af0-4429-abfc-165a45aefb07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:00.740360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1437/2896] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:14:00 2022] DELETE /identity/v3/projects/b7494caea46f45e691c06f5f29dc15d4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:00.745212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6b67e-4fcf-4dcf-8df6-554b75a58b48 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:00.745692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6b67e-4fcf-4dcf-8df6-554b75a58b48 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:00.745885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6b67e-4fcf-4dcf-8df6-554b75a58b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:00.765866 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-beb6b67e-4fcf-4dcf-8df6-554b75a58b48 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:00.767160 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-beb6b67e-4fcf-4dcf-8df6-554b75a58b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:00.831968 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1460/2897] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:00.841580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe60927-de90-4102-80da-0259d5021c70 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:00.841580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe60927-de90-4102-80da-0259d5021c70 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:00.841580 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe60927-de90-4102-80da-0259d5021c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:00.863110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-efe60927-de90-4102-80da-0259d5021c70 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:00.864573 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-efe60927-de90-4102-80da-0259d5021c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:00.873126 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1438/2898] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:00.878108 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6b67e-4fcf-4dcf-8df6-554b75a58b48 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:00.888541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6b67e-4fcf-4dcf-8df6-554b75a58b48 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:00.894699 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bbfd0-b7e9-4e70-96fa-a2bd7e4304ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:00.895317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bbfd0-b7e9-4e70-96fa-a2bd7e4304ee admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:00.895646 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bbfd0-b7e9-4e70-96fa-a2bd7e4304ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:00.895923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bbfd0-b7e9-4e70-96fa-a2bd7e4304ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:00.896631 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bbfd0-b7e9-4e70-96fa-a2bd7e4304ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:14:00.896929 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bbfd0-b7e9-4e70-96fa-a2bd7e4304ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:00.898530 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bbfd0-b7e9-4e70-96fa-a2bd7e4304ee admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:00.916481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1461/2899] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:14:00.921459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe60927-de90-4102-80da-0259d5021c70 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:00.932830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe60927-de90-4102-80da-0259d5021c70 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:00.938360 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a0e5f-ed97-47a2-8058-16e2c0b186ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:00.939080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a0e5f-ed97-47a2-8058-16e2c0b186ab admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:00.939381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a0e5f-ed97-47a2-8058-16e2c0b186ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:00.939653 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a0e5f-ed97-47a2-8058-16e2c0b186ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:00.940338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a0e5f-ed97-47a2-8058-16e2c0b186ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:00.942401 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a0e5f-ed97-47a2-8058-16e2c0b186ab admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:00.978195 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1439/2900] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:14:00 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:14:00.980430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bbfd0-b7e9-4e70-96fa-a2bd7e4304ee admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:00.992650 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bbfd0-b7e9-4e70-96fa-a2bd7e4304ee admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:01.022002 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:01.055050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8d754-89f8-477f-8a5a-bc9f903d881c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:01.055657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8d754-89f8-477f-8a5a-bc9f903d881c admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:01.055937 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8d754-89f8-477f-8a5a-bc9f903d881c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:01.056291 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8d754-89f8-477f-8a5a-bc9f903d881c admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:01.056926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8d754-89f8-477f-8a5a-bc9f903d881c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:01.058828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8d754-89f8-477f-8a5a-bc9f903d881c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:01.081573 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1462/2901] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:14:00 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:14:01.085847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a0e5f-ed97-47a2-8058-16e2c0b186ab admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:01.097319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a0e5f-ed97-47a2-8058-16e2c0b186ab admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:01.102052 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e871b7-ff2e-4733-9759-ac10e01c3c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:01.102651 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e871b7-ff2e-4733-9759-ac10e01c3c64 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:01.102926 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e871b7-ff2e-4733-9759-ac10e01c3c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:01.103204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e871b7-ff2e-4733-9759-ac10e01c3c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:01.103830 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e871b7-ff2e-4733-9759-ac10e01c3c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:01.105467 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e871b7-ff2e-4733-9759-ac10e01c3c64 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:01.118361 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1440/2902] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:01.120550 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8d754-89f8-477f-8a5a-bc9f903d881c admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:01.131093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8d754-89f8-477f-8a5a-bc9f903d881c admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:01.135831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830f16b-f736-43a8-8e18-0681262a5c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:01.136522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830f16b-f736-43a8-8e18-0681262a5c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:01.136807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830f16b-f736-43a8-8e18-0681262a5c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:01.137130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830f16b-f736-43a8-8e18-0681262a5c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:01.137777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830f16b-f736-43a8-8e18-0681262a5c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:01.139581 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830f16b-f736-43a8-8e18-0681262a5c7d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:01.151141 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1463/2903] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:14:01.155826 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e871b7-ff2e-4733-9759-ac10e01c3c64 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:01.168889 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e871b7-ff2e-4733-9759-ac10e01c3c64 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:01.174554 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811e99b-e71d-4c87-9d15-7fb6513de10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:01.175202 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811e99b-e71d-4c87-9d15-7fb6513de10c admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:01.175481 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811e99b-e71d-4c87-9d15-7fb6513de10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:01.175751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811e99b-e71d-4c87-9d15-7fb6513de10c admin admin] PATH_INFO: `/v3/projects/b17fe99b65514b4c99d8a8cc89cb130b/users/54aa5eb49f4a48e8bfc7a8cb069df9d6/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:01.191771 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811e99b-e71d-4c87-9d15-7fb6513de10c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b17fe99b65514b4c99d8a8cc89cb130b, user_id=54aa5eb49f4a48e8bfc7a8cb069df9d6, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:01.194602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811e99b-e71d-4c87-9d15-7fb6513de10c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:01.208850 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1441/2904] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:14:01 2022] PUT /identity/v3/projects/b17fe99b65514b4c99d8a8cc89cb130b/users/54aa5eb49f4a48e8bfc7a8cb069df9d6/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:01.222197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6bbff-5c2a-48d5-8081-e28cb4cf794f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:01.222482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6bbff-5c2a-48d5-8081-e28cb4cf794f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:01.222756 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6bbff-5c2a-48d5-8081-e28cb4cf794f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:01.256748 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-92a6bbff-5c2a-48d5-8081-e28cb4cf794f None None] MFA Rules not processed for user `54aa5eb49f4a48e8bfc7a8cb069df9d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:01.257989 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-92a6bbff-5c2a-48d5-8081-e28cb4cf794f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:01.324803 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1464/2905] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:01.329471 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811e99b-e71d-4c87-9d15-7fb6513de10c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:01.393794 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811e99b-e71d-4c87-9d15-7fb6513de10c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:01.399021 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec0149-4f50-471b-84bf-7c2f77f80f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:01.399595 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec0149-4f50-471b-84bf-7c2f77f80f73 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:01.399880 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec0149-4f50-471b-84bf-7c2f77f80f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:01.400205 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec0149-4f50-471b-84bf-7c2f77f80f73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:01.400861 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec0149-4f50-471b-84bf-7c2f77f80f73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:01.402786 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec0149-4f50-471b-84bf-7c2f77f80f73 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:01.431531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1442/2906] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:14:01 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:14:01.436339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6bbff-5c2a-48d5-8081-e28cb4cf794f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:01.447302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6bbff-5c2a-48d5-8081-e28cb4cf794f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:01.504746 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e2444-f041-4680-8006-6ca6f0ac46bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:01.505371 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e2444-f041-4680-8006-6ca6f0ac46bf admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:01.505654 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e2444-f041-4680-8006-6ca6f0ac46bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:01.505927 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e2444-f041-4680-8006-6ca6f0ac46bf admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:01.506565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e2444-f041-4680-8006-6ca6f0ac46bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:01.508566 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e2444-f041-4680-8006-6ca6f0ac46bf admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:01.534529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1465/2907] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:14:01 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:14:01.538593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec0149-4f50-471b-84bf-7c2f77f80f73 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:01.549611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec0149-4f50-471b-84bf-7c2f77f80f73 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:01.554506 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c217d-f8fe-4899-ab18-ba0eacb61ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:01.555110 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c217d-f8fe-4899-ab18-ba0eacb61ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:01.555391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c217d-f8fe-4899-ab18-ba0eacb61ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:01.555665 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c217d-f8fe-4899-ab18-ba0eacb61ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:01.556355 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c217d-f8fe-4899-ab18-ba0eacb61ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:01.558112 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c217d-f8fe-4899-ab18-ba0eacb61ac1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:01.569226 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1443/2908] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:01.573177 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e2444-f041-4680-8006-6ca6f0ac46bf admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:01.585347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e2444-f041-4680-8006-6ca6f0ac46bf admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:01.590058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66243ddb-2d23-444e-b5dd-ac690aaef90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:01.590656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66243ddb-2d23-444e-b5dd-ac690aaef90c admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:01.590936 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66243ddb-2d23-444e-b5dd-ac690aaef90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:01.591204 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66243ddb-2d23-444e-b5dd-ac690aaef90c admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:01.592491 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66243ddb-2d23-444e-b5dd-ac690aaef90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:01.594465 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66243ddb-2d23-444e-b5dd-ac690aaef90c admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:01.604983 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1466/2909] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:01.609064 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c217d-f8fe-4899-ab18-ba0eacb61ac1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:01.619636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c217d-f8fe-4899-ab18-ba0eacb61ac1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:01.624832 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042761b-3e43-444c-bb94-ea01af8639da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:01.625488 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042761b-3e43-444c-bb94-ea01af8639da admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:01.625766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042761b-3e43-444c-bb94-ea01af8639da admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:01.626036 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042761b-3e43-444c-bb94-ea01af8639da admin admin] PATH_INFO: `/v3/projects/c7e91a164edf48f5a41d6d589e4592d4/users/4f1937f1efca48ada9f5f09faa2bbbb5/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:01.642068 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042761b-3e43-444c-bb94-ea01af8639da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7e91a164edf48f5a41d6d589e4592d4, user_id=4f1937f1efca48ada9f5f09faa2bbbb5, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:01.644829 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042761b-3e43-444c-bb94-ea01af8639da admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:01.655049 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:01.661499 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1444/2910] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:14:01 2022] PUT /identity/v3/projects/c7e91a164edf48f5a41d6d589e4592d4/users/4f1937f1efca48ada9f5f09faa2bbbb5/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:01.672664 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c1619-a7e6-408f-bdd3-f4da8533960a None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:01.672959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c1619-a7e6-408f-bdd3-f4da8533960a None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:01.673261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c1619-a7e6-408f-bdd3-f4da8533960a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:01.705166 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-bd7c1619-a7e6-408f-bdd3-f4da8533960a None None] MFA Rules not processed for user `4f1937f1efca48ada9f5f09faa2bbbb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:01.706478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-bd7c1619-a7e6-408f-bdd3-f4da8533960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:01.710565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:01.774972 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1467/2911] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:01.780023 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042761b-3e43-444c-bb94-ea01af8639da admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:01.849381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042761b-3e43-444c-bb94-ea01af8639da admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:01.854290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae05e71-b536-4fa5-869e-1c9122fcdc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:01.854950 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae05e71-b536-4fa5-869e-1c9122fcdc61 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:01.855234 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae05e71-b536-4fa5-869e-1c9122fcdc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:01.855507 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae05e71-b536-4fa5-869e-1c9122fcdc61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:01.856186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae05e71-b536-4fa5-869e-1c9122fcdc61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:01.858753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae05e71-b536-4fa5-869e-1c9122fcdc61 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:01.887759 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1445/2912] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:14:01 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:14:01.893115 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c1619-a7e6-408f-bdd3-f4da8533960a None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:01.910626 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c1619-a7e6-408f-bdd3-f4da8533960a None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:01.970830 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736626a-f0e1-4977-973b-fd043ae5ba6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:01.971450 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736626a-f0e1-4977-973b-fd043ae5ba6a admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:01.971728 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736626a-f0e1-4977-973b-fd043ae5ba6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:01.972015 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736626a-f0e1-4977-973b-fd043ae5ba6a admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:01.972651 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736626a-f0e1-4977-973b-fd043ae5ba6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:01.974899 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736626a-f0e1-4977-973b-fd043ae5ba6a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:01.997438 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1468/2913] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:14:01 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:14:02.002071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae05e71-b536-4fa5-869e-1c9122fcdc61 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:02.012987 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae05e71-b536-4fa5-869e-1c9122fcdc61 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:02.018732 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d315742-d2e5-4bfd-968c-156e48ca6515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:02.019400 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315742-d2e5-4bfd-968c-156e48ca6515 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:02.019691 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315742-d2e5-4bfd-968c-156e48ca6515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:02.019941 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315742-d2e5-4bfd-968c-156e48ca6515 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:02.020625 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d315742-d2e5-4bfd-968c-156e48ca6515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:02.022283 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d315742-d2e5-4bfd-968c-156e48ca6515 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:02.034217 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1446/2914] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:02.038431 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736626a-f0e1-4977-973b-fd043ae5ba6a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:02.049804 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736626a-f0e1-4977-973b-fd043ae5ba6a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:02.058574 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101349ab-e0d7-4858-886f-22694d432829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:02.059259 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101349ab-e0d7-4858-886f-22694d432829 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:02.059538 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101349ab-e0d7-4858-886f-22694d432829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:02.059912 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101349ab-e0d7-4858-886f-22694d432829 admin admin] PATH_INFO: `/v3/projects/8e35b14d62d84fcc85d2f51e20455349/users/bff5c8a0ce454ee19645947179ea11c9/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:02.077885 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101349ab-e0d7-4858-886f-22694d432829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e35b14d62d84fcc85d2f51e20455349, user_id=bff5c8a0ce454ee19645947179ea11c9, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:02.080695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101349ab-e0d7-4858-886f-22694d432829 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:02.097942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1469/2915] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:14:02 2022] PUT /identity/v3/projects/8e35b14d62d84fcc85d2f51e20455349/users/bff5c8a0ce454ee19645947179ea11c9/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:02.102016 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d315742-d2e5-4bfd-968c-156e48ca6515 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:02.113787 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d315742-d2e5-4bfd-968c-156e48ca6515 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:02.178472 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a9a64-6b83-422c-a63c-fa0a41a5cef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:02.179145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a9a64-6b83-422c-a63c-fa0a41a5cef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:02.179487 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a9a64-6b83-422c-a63c-fa0a41a5cef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:02.179823 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a9a64-6b83-422c-a63c-fa0a41a5cef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:02.180623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a9a64-6b83-422c-a63c-fa0a41a5cef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:02.183764 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a9a64-6b83-422c-a63c-fa0a41a5cef6 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:02.199161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1447/2916] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:02.200771 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101349ab-e0d7-4858-886f-22694d432829 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:02.214119 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101349ab-e0d7-4858-886f-22694d432829 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:02.219430 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9af95-c525-4568-b647-a2510340ab18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:02.220213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9af95-c525-4568-b647-a2510340ab18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:02.220562 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9af95-c525-4568-b647-a2510340ab18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:02.220905 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9af95-c525-4568-b647-a2510340ab18 admin admin] PATH_INFO: `/v3/projects/8e35b14d62d84fcc85d2f51e20455349/users/bff5c8a0ce454ee19645947179ea11c9/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:02.222872 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9af95-c525-4568-b647-a2510340ab18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e35b14d62d84fcc85d2f51e20455349, user_id=bff5c8a0ce454ee19645947179ea11c9, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:02.225591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9af95-c525-4568-b647-a2510340ab18 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:02.236998 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1470/2917] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:14:02 2022] PUT /identity/v3/projects/8e35b14d62d84fcc85d2f51e20455349/users/bff5c8a0ce454ee19645947179ea11c9/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:02.247457 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae010e-9af5-4d2b-bd0b-f4522671a3f3 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:02.247741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae010e-9af5-4d2b-bd0b-f4522671a3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:02.248707 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae010e-9af5-4d2b-bd0b-f4522671a3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:02.277019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-d9ae010e-9af5-4d2b-bd0b-f4522671a3f3 None None] MFA Rules not processed for user `bff5c8a0ce454ee19645947179ea11c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:02.278227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d9ae010e-9af5-4d2b-bd0b-f4522671a3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:02.339956 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:02.351062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1448/2918] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:02.362856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9af95-c525-4568-b647-a2510340ab18 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:02.364648 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2dd9af95-c525-4568-b647-a2510340ab18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:02.422778 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9af95-c525-4568-b647-a2510340ab18 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:02.426461 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd365485-16c0-472e-877e-32b84d9d7cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:02.427059 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd365485-16c0-472e-877e-32b84d9d7cbf service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:02.427340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd365485-16c0-472e-877e-32b84d9d7cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:02.427614 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd365485-16c0-472e-877e-32b84d9d7cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:02.437263 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd365485-16c0-472e-877e-32b84d9d7cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:02.438721 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd365485-16c0-472e-877e-32b84d9d7cbf service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:02.443743 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:02.456229 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1471/2919] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:02 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:02.976124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae010e-9af5-4d2b-bd0b-f4522671a3f3 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:02.977541 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-d9ae010e-9af5-4d2b-bd0b-f4522671a3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:02.991070 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae010e-9af5-4d2b-bd0b-f4522671a3f3 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:02.994799 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414b886-675c-481b-99d3-e0072f83c53f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:02.995407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414b886-675c-481b-99d3-e0072f83c53f service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:02.995688 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414b886-675c-481b-99d3-e0072f83c53f service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:02.995962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414b886-675c-481b-99d3-e0072f83c53f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:03.055696 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414b886-675c-481b-99d3-e0072f83c53f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:03.057264 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414b886-675c-481b-99d3-e0072f83c53f service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:03.067092 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1449/2920] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:07.200230 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd365485-16c0-472e-877e-32b84d9d7cbf service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:07.210044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd365485-16c0-472e-877e-32b84d9d7cbf service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:07.214292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019c6ef-21e9-4568-a5b9-25db25b89b7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:07.214900 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019c6ef-21e9-4568-a5b9-25db25b89b7c service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:07.215185 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019c6ef-21e9-4568-a5b9-25db25b89b7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:07.215456 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019c6ef-21e9-4568-a5b9-25db25b89b7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:07.275347 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019c6ef-21e9-4568-a5b9-25db25b89b7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:07.276757 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9019c6ef-21e9-4568-a5b9-25db25b89b7c service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:07.286519 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1472/2921] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:09.248153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414b886-675c-481b-99d3-e0072f83c53f service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:09.250381 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c414b886-675c-481b-99d3-e0072f83c53f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:09.317789 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414b886-675c-481b-99d3-e0072f83c53f service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:09.321801 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1ee80-c393-4e1e-85e0-13a2412219dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:09.322611 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1ee80-c393-4e1e-85e0-13a2412219dd service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:09.323004 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1ee80-c393-4e1e-85e0-13a2412219dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:09.323497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1ee80-c393-4e1e-85e0-13a2412219dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:09.325600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8de1ee80-c393-4e1e-85e0-13a2412219dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:09.408067 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1ee80-c393-4e1e-85e0-13a2412219dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:09.409527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1ee80-c393-4e1e-85e0-13a2412219dd service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:09.421577 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1450/2922] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:14:09.489394 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019c6ef-21e9-4568-a5b9-25db25b89b7c service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:09.554277 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019c6ef-21e9-4568-a5b9-25db25b89b7c service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:09.558881 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:09.559585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:09.559932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:09.560906 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] PATH_INFO: `/v3/users/54aa5eb49f4a48e8bfc7a8cb069df9d6` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:09.562260 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54aa5eb49f4a48e8bfc7a8cb069df9d6)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:09.564304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:09.611664 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54aa5eb49f4a48e8bfc7a8cb069df9d6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:09.612301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54aa5eb49f4a48e8bfc7a8cb069df9d6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:09.619401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54aa5eb49f4a48e8bfc7a8cb069df9d6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:09.627657 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54aa5eb49f4a48e8bfc7a8cb069df9d6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:09.640493 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54aa5eb49f4a48e8bfc7a8cb069df9d6'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:09.646759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1473/2923] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:14:09 2022] DELETE /identity/v3/users/54aa5eb49f4a48e8bfc7a8cb069df9d6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:09.651714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f896e-b301-4907-b123-ca0778f2c047 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:09.652019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f896e-b301-4907-b123-ca0778f2c047 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:09.652322 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f896e-b301-4907-b123-ca0778f2c047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:09.671767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-be2f896e-b301-4907-b123-ca0778f2c047 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:09.673022 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-be2f896e-b301-4907-b123-ca0778f2c047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:09.703435 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=100 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:09.733995 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1451/2924] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:14:09.743655 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:09.745178 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:09.766801 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:09.807473 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29343d2-b876-40bd-abdf-600c49b7b771 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:09.811207 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9460f3-e9c1-4e76-8724-1c1a57a5c322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:09.811813 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9460f3-e9c1-4e76-8724-1c1a57a5c322 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:09.812136 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9460f3-e9c1-4e76-8724-1c1a57a5c322 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:09.812405 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9460f3-e9c1-4e76-8724-1c1a57a5c322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:09.821635 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9460f3-e9c1-4e76-8724-1c1a57a5c322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:09.823053 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9460f3-e9c1-4e76-8724-1c1a57a5c322 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:09.834007 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1474/2925] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:10.106659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f896e-b301-4907-b123-ca0778f2c047 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:10.108455 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-be2f896e-b301-4907-b123-ca0778f2c047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:10.123430 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f896e-b301-4907-b123-ca0778f2c047 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:10.128895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:10.129610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:10.129962 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:10.130325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] PATH_INFO: `/v3/projects/b17fe99b65514b4c99d8a8cc89cb130b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:10.131516 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b17fe99b65514b4c99d8a8cc89cb130b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:10.133538 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:10.182145 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b17fe99b65514b4c99d8a8cc89cb130b', 'request_id': 'req-15e80919-b523-47af-bca9-cce576af6c83'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.191566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b17fe99b65514b4c99d8a8cc89cb130b', 'request_id': 'req-15e80919-b523-47af-bca9-cce576af6c83'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.199339 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b17fe99b65514b4c99d8a8cc89cb130b', 'request_id': 'req-15e80919-b523-47af-bca9-cce576af6c83'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.210774 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] The token cache is being invalidate because project b17fe99b65514b4c99d8a8cc89cb130b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:14:10.211103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.211770 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.213338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1452/2926] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:14:10 2022] DELETE /identity/v3/projects/b17fe99b65514b4c99d8a8cc89cb130b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:10.218286 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9460f3-e9c1-4e76-8724-1c1a57a5c322 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:10.227967 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9460f3-e9c1-4e76-8724-1c1a57a5c322 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:10.289309 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:10.290448 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:10.290889 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:10.291317 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] PATH_INFO: `/v3/users/4f1937f1efca48ada9f5f09faa2bbbb5` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:10.292641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f1937f1efca48ada9f5f09faa2bbbb5)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:10.295311 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:10.360212 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f1937f1efca48ada9f5f09faa2bbbb5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.360911 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f1937f1efca48ada9f5f09faa2bbbb5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.367974 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f1937f1efca48ada9f5f09faa2bbbb5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.376627 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f1937f1efca48ada9f5f09faa2bbbb5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.390720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f1937f1efca48ada9f5f09faa2bbbb5'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.397184 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1475/2927] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:14:10 2022] DELETE /identity/v3/users/4f1937f1efca48ada9f5f09faa2bbbb5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:10.708257 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:10.709922 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:10.744783 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e80919-b523-47af-bca9-cce576af6c83 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:10.809766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:10.811318 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:10.811915 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:10.812349 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] PATH_INFO: `/v3/projects/c7e91a164edf48f5a41d6d589e4592d4` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:10.813649 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7e91a164edf48f5a41d6d589e4592d4)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:10.815724 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:10.871686 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7e91a164edf48f5a41d6d589e4592d4', 'request_id': 'req-79c90d31-6531-4386-a989-dd30323d88e3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.881038 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7e91a164edf48f5a41d6d589e4592d4', 'request_id': 'req-79c90d31-6531-4386-a989-dd30323d88e3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.887916 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7e91a164edf48f5a41d6d589e4592d4', 'request_id': 'req-79c90d31-6531-4386-a989-dd30323d88e3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.890224 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:10.902629 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] The token cache is being invalidate because project c7e91a164edf48f5a41d6d589e4592d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:14:10.903502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.904459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:10.906822 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1453/2928] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:14:10 2022] DELETE /identity/v3/projects/c7e91a164edf48f5a41d6d589e4592d4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:10.910893 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:10.912355 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:10.930109 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d96dae-a947-4dfe-a765-9b687f5c3288 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:10.969175 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:10.988401 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf20cde-eb0e-4e4e-8ab4-38886e62465a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:10.989159 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf20cde-eb0e-4e4e-8ab4-38886e62465a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:10.989442 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf20cde-eb0e-4e4e-8ab4-38886e62465a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:10.989712 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf20cde-eb0e-4e4e-8ab4-38886e62465a admin admin] PATH_INFO: `/v3/users/bff5c8a0ce454ee19645947179ea11c9` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:10.990829 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf20cde-eb0e-4e4e-8ab4-38886e62465a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bff5c8a0ce454ee19645947179ea11c9)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:10.992781 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf20cde-eb0e-4e4e-8ab4-38886e62465a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:11.041537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aaf20cde-eb0e-4e4e-8ab4-38886e62465a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bff5c8a0ce454ee19645947179ea11c9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:11.042174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aaf20cde-eb0e-4e4e-8ab4-38886e62465a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bff5c8a0ce454ee19645947179ea11c9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:11.049923 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aaf20cde-eb0e-4e4e-8ab4-38886e62465a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bff5c8a0ce454ee19645947179ea11c9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:11.057323 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aaf20cde-eb0e-4e4e-8ab4-38886e62465a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bff5c8a0ce454ee19645947179ea11c9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:11.070496 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-aaf20cde-eb0e-4e4e-8ab4-38886e62465a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bff5c8a0ce454ee19645947179ea11c9'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:11.076725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1476/2929] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:14:10 2022] DELETE /identity/v3/users/bff5c8a0ce454ee19645947179ea11c9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:11.369436 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:11.373466 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:11.386900 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c90d31-6531-4386-a989-dd30323d88e3 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:11.446330 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7449f-7aad-444d-ac81-e603788e6fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:11.446881 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7449f-7aad-444d-ac81-e603788e6fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:11.447164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7449f-7aad-444d-ac81-e603788e6fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:11.447438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7449f-7aad-444d-ac81-e603788e6fb3 admin admin] PATH_INFO: `/v3/projects/8e35b14d62d84fcc85d2f51e20455349` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:11.448584 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7449f-7aad-444d-ac81-e603788e6fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e35b14d62d84fcc85d2f51e20455349)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:11.450515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7449f-7aad-444d-ac81-e603788e6fb3 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:11.500080 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6fb7449f-7aad-444d-ac81-e603788e6fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e35b14d62d84fcc85d2f51e20455349', 'request_id': 'req-6fb7449f-7aad-444d-ac81-e603788e6fb3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:11.508512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6fb7449f-7aad-444d-ac81-e603788e6fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e35b14d62d84fcc85d2f51e20455349', 'request_id': 'req-6fb7449f-7aad-444d-ac81-e603788e6fb3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:11.515129 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6fb7449f-7aad-444d-ac81-e603788e6fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e35b14d62d84fcc85d2f51e20455349', 'request_id': 'req-6fb7449f-7aad-444d-ac81-e603788e6fb3'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:11.525993 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6fb7449f-7aad-444d-ac81-e603788e6fb3 admin admin] The token cache is being invalidate because project 8e35b14d62d84fcc85d2f51e20455349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:14:11.526284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6fb7449f-7aad-444d-ac81-e603788e6fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:11.526914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6fb7449f-7aad-444d-ac81-e603788e6fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:11.528464 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1454/2930] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:14:11 2022] DELETE /identity/v3/projects/8e35b14d62d84fcc85d2f51e20455349 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:11.534980 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239361f7-3c3b-49cf-bcc7-6ed5a24a3c3e None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:11.535267 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239361f7-3c3b-49cf-bcc7-6ed5a24a3c3e None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:11.535545 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239361f7-3c3b-49cf-bcc7-6ed5a24a3c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:11.555203 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-239361f7-3c3b-49cf-bcc7-6ed5a24a3c3e None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:11.556445 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-239361f7-3c3b-49cf-bcc7-6ed5a24a3c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:11.618957 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1477/2931] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:11.627918 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6712d-d759-4346-b68b-e561860e259c None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:11.628314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6712d-d759-4346-b68b-e561860e259c None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:11.628591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6712d-d759-4346-b68b-e561860e259c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:11.648196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-e7c6712d-d759-4346-b68b-e561860e259c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:11.649680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-e7c6712d-d759-4346-b68b-e561860e259c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:11.658111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1455/2932] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:11.662322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239361f7-3c3b-49cf-bcc7-6ed5a24a3c3e None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:11.672529 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239361f7-3c3b-49cf-bcc7-6ed5a24a3c3e None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:11.677051 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795aa4e-8b31-4de2-9820-54f20663d38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:11.677645 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795aa4e-8b31-4de2-9820-54f20663d38a admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:11.677928 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795aa4e-8b31-4de2-9820-54f20663d38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:11.678198 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795aa4e-8b31-4de2-9820-54f20663d38a admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:11.678835 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795aa4e-8b31-4de2-9820-54f20663d38a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:14:11.679137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795aa4e-8b31-4de2-9820-54f20663d38a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:11.680785 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795aa4e-8b31-4de2-9820-54f20663d38a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:11.692790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1478/2933] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 06:14:11.697117 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6712d-d759-4346-b68b-e561860e259c None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:11.706570 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6712d-d759-4346-b68b-e561860e259c None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:11.711236 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80597461-7668-4a0d-822a-c1b8eff5e63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:11.711849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80597461-7668-4a0d-822a-c1b8eff5e63b admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:11.712133 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80597461-7668-4a0d-822a-c1b8eff5e63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:11.712407 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80597461-7668-4a0d-822a-c1b8eff5e63b admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:11.713043 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80597461-7668-4a0d-822a-c1b8eff5e63b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:11.714946 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80597461-7668-4a0d-822a-c1b8eff5e63b admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:11.747183 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1456/2934] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:14:11 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:14:11.748503 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795aa4e-8b31-4de2-9820-54f20663d38a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:11.758738 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795aa4e-8b31-4de2-9820-54f20663d38a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:11.809978 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:11.824823 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7703c7-dd7c-487c-ae12-cae9129c5812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:11.825440 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7703c7-dd7c-487c-ae12-cae9129c5812 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:11.825720 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7703c7-dd7c-487c-ae12-cae9129c5812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:11.826752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7703c7-dd7c-487c-ae12-cae9129c5812 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:11.827453 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7703c7-dd7c-487c-ae12-cae9129c5812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:11.829459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7703c7-dd7c-487c-ae12-cae9129c5812 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:11.859090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1479/2935] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:14:11 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:14:11.863930 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80597461-7668-4a0d-822a-c1b8eff5e63b admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:11.878636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80597461-7668-4a0d-822a-c1b8eff5e63b admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:11.883739 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63686032-2526-4769-b527-e5712cc2ec76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:11.884382 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686032-2526-4769-b527-e5712cc2ec76 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:11.884666 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686032-2526-4769-b527-e5712cc2ec76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:11.884937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686032-2526-4769-b527-e5712cc2ec76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:11.885655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63686032-2526-4769-b527-e5712cc2ec76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:11.887290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63686032-2526-4769-b527-e5712cc2ec76 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:11.913543 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1457/2936] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:11.917462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7703c7-dd7c-487c-ae12-cae9129c5812 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:11.938093 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7703c7-dd7c-487c-ae12-cae9129c5812 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:11.942859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac47ac2-643f-4df2-8ec0-8769fb060a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:11.943963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac47ac2-643f-4df2-8ec0-8769fb060a8e admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:11.943963 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac47ac2-643f-4df2-8ec0-8769fb060a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:11.944261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac47ac2-643f-4df2-8ec0-8769fb060a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:11.945222 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac47ac2-643f-4df2-8ec0-8769fb060a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:11.946701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac47ac2-643f-4df2-8ec0-8769fb060a8e admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:11.962382 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1480/2937] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:11.963098 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63686032-2526-4769-b527-e5712cc2ec76 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:11.973898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63686032-2526-4769-b527-e5712cc2ec76 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:11.978778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393383f4-04af-4477-8f09-16a9d58320eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:11.979532 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393383f4-04af-4477-8f09-16a9d58320eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:11.979914 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393383f4-04af-4477-8f09-16a9d58320eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:11.980319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393383f4-04af-4477-8f09-16a9d58320eb admin admin] PATH_INFO: `/v3/projects/071d63d0edff418d90b93a9caabf7358/users/ba62460d44a043bab1f99be1897bcd74/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:11.998221 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393383f4-04af-4477-8f09-16a9d58320eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=071d63d0edff418d90b93a9caabf7358, user_id=ba62460d44a043bab1f99be1897bcd74, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:12.001246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393383f4-04af-4477-8f09-16a9d58320eb admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:12.016476 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1458/2938] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:14:11 2022] PUT /identity/v3/projects/071d63d0edff418d90b93a9caabf7358/users/ba62460d44a043bab1f99be1897bcd74/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:12.041031 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cacca5-6456-48c4-a83c-c38d505aec5d None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:12.041313 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cacca5-6456-48c4-a83c-c38d505aec5d None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:12.041587 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cacca5-6456-48c4-a83c-c38d505aec5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:12.072208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-a2cacca5-6456-48c4-a83c-c38d505aec5d None None] MFA Rules not processed for user `ba62460d44a043bab1f99be1897bcd74`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:12.073428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-a2cacca5-6456-48c4-a83c-c38d505aec5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:12.137248 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1481/2939] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:12 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:12.141849 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393383f4-04af-4477-8f09-16a9d58320eb admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:12.160635 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:12.209253 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393383f4-04af-4477-8f09-16a9d58320eb admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:12.214146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c786adf-48ed-468d-96a8-8721ee8e943a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:12.214687 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c786adf-48ed-468d-96a8-8721ee8e943a admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:12.214961 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c786adf-48ed-468d-96a8-8721ee8e943a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:12.215227 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c786adf-48ed-468d-96a8-8721ee8e943a admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:12.215847 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c786adf-48ed-468d-96a8-8721ee8e943a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:12.218409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c786adf-48ed-468d-96a8-8721ee8e943a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:12.245136 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1459/2940] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:14:12 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:14:12.250019 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cacca5-6456-48c4-a83c-c38d505aec5d None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:12.260856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cacca5-6456-48c4-a83c-c38d505aec5d None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:12.319537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352a0c8-a9b3-4cf3-8767-80d41c494502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:12.320182 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352a0c8-a9b3-4cf3-8767-80d41c494502 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:12.320459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352a0c8-a9b3-4cf3-8767-80d41c494502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:12.320727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352a0c8-a9b3-4cf3-8767-80d41c494502 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:12.321510 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352a0c8-a9b3-4cf3-8767-80d41c494502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:12.323422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352a0c8-a9b3-4cf3-8767-80d41c494502 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:12.345942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1482/2941] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:14:12 2022] POST /identity/v3/users => generated 307 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:14:12.349593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c786adf-48ed-468d-96a8-8721ee8e943a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:12.360275 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c786adf-48ed-468d-96a8-8721ee8e943a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:12.364865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec032267-9c2b-4a67-9e65-5b4a7294e329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:12.365569 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec032267-9c2b-4a67-9e65-5b4a7294e329 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:12.365845 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec032267-9c2b-4a67-9e65-5b4a7294e329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:12.366111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec032267-9c2b-4a67-9e65-5b4a7294e329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:12.366767 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec032267-9c2b-4a67-9e65-5b4a7294e329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:12.368398 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec032267-9c2b-4a67-9e65-5b4a7294e329 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:12.380772 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1460/2942] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 06:14:12.384932 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352a0c8-a9b3-4cf3-8767-80d41c494502 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:12.395388 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352a0c8-a9b3-4cf3-8767-80d41c494502 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:12.400303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1ff5e-87da-4105-a008-e0caeb6b199f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:12.400874 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ff5e-87da-4105-a008-e0caeb6b199f admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:12.401150 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ff5e-87da-4105-a008-e0caeb6b199f admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:12.401421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ff5e-87da-4105-a008-e0caeb6b199f admin admin] PATH_INFO: `/v3/projects/cb503c7069d9423a9e7a24603c62d56e/users/db224a6060b2420db998d6d9c56ad9fa/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:12.421877 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1ff5e-87da-4105-a008-e0caeb6b199f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb503c7069d9423a9e7a24603c62d56e, user_id=db224a6060b2420db998d6d9c56ad9fa, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:12.424610 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1ff5e-87da-4105-a008-e0caeb6b199f admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:12.437578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1483/2943] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:14:12 2022] PUT /identity/v3/projects/cb503c7069d9423a9e7a24603c62d56e/users/db224a6060b2420db998d6d9c56ad9fa/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:14:12.441622 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec032267-9c2b-4a67-9e65-5b4a7294e329 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:12.451936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec032267-9c2b-4a67-9e65-5b4a7294e329 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:12.511953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff3fb8-4c19-4527-a371-6d96eb5f2fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:12.512591 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff3fb8-4c19-4527-a371-6d96eb5f2fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:12.512932 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff3fb8-4c19-4527-a371-6d96eb5f2fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:12.513898 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff3fb8-4c19-4527-a371-6d96eb5f2fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:12.514417 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff3fb8-4c19-4527-a371-6d96eb5f2fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:12.516287 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff3fb8-4c19-4527-a371-6d96eb5f2fb0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:12.527190 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1461/2944] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:12.531686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1ff5e-87da-4105-a008-e0caeb6b199f admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:12.542735 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1ff5e-87da-4105-a008-e0caeb6b199f admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:12.548578 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4543c6d-18cd-4656-b393-7385e686b6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:12.549365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4543c6d-18cd-4656-b393-7385e686b6d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:12.549707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4543c6d-18cd-4656-b393-7385e686b6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:12.550044 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4543c6d-18cd-4656-b393-7385e686b6d0 admin admin] PATH_INFO: `/v3/projects/cb503c7069d9423a9e7a24603c62d56e/users/db224a6060b2420db998d6d9c56ad9fa/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:12.552045 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4543c6d-18cd-4656-b393-7385e686b6d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb503c7069d9423a9e7a24603c62d56e, user_id=db224a6060b2420db998d6d9c56ad9fa, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:12.554777 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4543c6d-18cd-4656-b393-7385e686b6d0 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:12.568338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1484/2945] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:14:12 2022] PUT /identity/v3/projects/cb503c7069d9423a9e7a24603c62d56e/users/db224a6060b2420db998d6d9c56ad9fa/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:12.578605 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae6f39-8cc9-4f0b-894c-0f951893beec None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:12.579193 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae6f39-8cc9-4f0b-894c-0f951893beec None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:12.580233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae6f39-8cc9-4f0b-894c-0f951893beec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:12.617124 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-62ae6f39-8cc9-4f0b-894c-0f951893beec None None] MFA Rules not processed for user `db224a6060b2420db998d6d9c56ad9fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:12.619529 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-62ae6f39-8cc9-4f0b-894c-0f951893beec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:12.704201 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1462/2946] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:12 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:14:12.725731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4543c6d-18cd-4656-b393-7385e686b6d0 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:12.728013 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-f4543c6d-18cd-4656-b393-7385e686b6d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:12.742167 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:12.804400 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4543c6d-18cd-4656-b393-7385e686b6d0 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:12.808221 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752e598-ae71-4aa9-b90b-6514f9351f00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:12.808807 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752e598-ae71-4aa9-b90b-6514f9351f00 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:12.809090 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752e598-ae71-4aa9-b90b-6514f9351f00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:12.809365 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752e598-ae71-4aa9-b90b-6514f9351f00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:12.818964 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752e598-ae71-4aa9-b90b-6514f9351f00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:12.820435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752e598-ae71-4aa9-b90b-6514f9351f00 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:12.830891 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1485/2947] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:12 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:12.916044 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae6f39-8cc9-4f0b-894c-0f951893beec None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:12.927041 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae6f39-8cc9-4f0b-894c-0f951893beec None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:12.931213 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c071595-d130-42e7-aa19-c69c91ca3090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:12.932070 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c071595-d130-42e7-aa19-c69c91ca3090 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:12.932425 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c071595-d130-42e7-aa19-c69c91ca3090 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:12.932868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c071595-d130-42e7-aa19-c69c91ca3090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:12.943414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:13.004254 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c071595-d130-42e7-aa19-c69c91ca3090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:13.005982 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c071595-d130-42e7-aa19-c69c91ca3090 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:13.017109 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1463/2948] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:13.372314 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752e598-ae71-4aa9-b90b-6514f9351f00 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:13.374297 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-3752e598-ae71-4aa9-b90b-6514f9351f00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:13.448479 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752e598-ae71-4aa9-b90b-6514f9351f00 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:13.453254 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccaef6-322e-41c1-a949-22e51a48fb42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:13.453979 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccaef6-322e-41c1-a949-22e51a48fb42 service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:13.454482 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccaef6-322e-41c1-a949-22e51a48fb42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:13.454860 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccaef6-322e-41c1-a949-22e51a48fb42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:13.456888 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c5ccaef6-322e-41c1-a949-22e51a48fb42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:13.467682 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccaef6-322e-41c1-a949-22e51a48fb42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:13.469287 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccaef6-322e-41c1-a949-22e51a48fb42 service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:13.479409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1486/2949] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:13.616986 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c071595-d130-42e7-aa19-c69c91ca3090 service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:13.627054 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c071595-d130-42e7-aa19-c69c91ca3090 service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:13.631074 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a102f-255f-41f1-a693-1016e3310f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:13.631677 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a102f-255f-41f1-a693-1016e3310f1c service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:13.631959 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a102f-255f-41f1-a693-1016e3310f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:13.632288 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a102f-255f-41f1-a693-1016e3310f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:13.633996 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-993a102f-255f-41f1-a693-1016e3310f1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:13.645372 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a102f-255f-41f1-a693-1016e3310f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:13.647852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a102f-255f-41f1-a693-1016e3310f1c service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:13.658862 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1464/2950] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:13.741838 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccaef6-322e-41c1-a949-22e51a48fb42 service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:13.752427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccaef6-322e-41c1-a949-22e51a48fb42 service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:13.768195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b014f-6d3d-4d1f-bcba-e74f442afe6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:13.768195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b014f-6d3d-4d1f-bcba-e74f442afe6b service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:13.768195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b014f-6d3d-4d1f-bcba-e74f442afe6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:13.768195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b014f-6d3d-4d1f-bcba-e74f442afe6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:13.768642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-aa7b014f-6d3d-4d1f-bcba-e74f442afe6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:13.836421 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b014f-6d3d-4d1f-bcba-e74f442afe6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:13.837875 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b014f-6d3d-4d1f-bcba-e74f442afe6b service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:13.849220 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1487/2951] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:13.913890 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a102f-255f-41f1-a693-1016e3310f1c service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:13.915325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-993a102f-255f-41f1-a693-1016e3310f1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:13.996408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a102f-255f-41f1-a693-1016e3310f1c service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:14.002290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd964e8-0800-44e7-ba36-c0c6f3a94f39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:14.002912 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd964e8-0800-44e7-ba36-c0c6f3a94f39 service placement] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:14.003186 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd964e8-0800-44e7-ba36-c0c6f3a94f39 service placement] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:14.003478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd964e8-0800-44e7-ba36-c0c6f3a94f39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:14.005177 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-1fd964e8-0800-44e7-ba36-c0c6f3a94f39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:14.017204 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd964e8-0800-44e7-ba36-c0c6f3a94f39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:14.018773 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd964e8-0800-44e7-ba36-c0c6f3a94f39 service placement] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:14.028949 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1465/2952] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:16.647130 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b014f-6d3d-4d1f-bcba-e74f442afe6b service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:16.656302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-aa7b014f-6d3d-4d1f-bcba-e74f442afe6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:16.673474 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b014f-6d3d-4d1f-bcba-e74f442afe6b service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:16.680449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da9f46-4ee8-4ded-98c3-af59090d232f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bd5a9a09f034c05815c631a44d1cdd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:16.681069 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da9f46-4ee8-4ded-98c3-af59090d232f service nova] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:16.681702 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da9f46-4ee8-4ded-98c3-af59090d232f service nova] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:16.681988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da9f46-4ee8-4ded-98c3-af59090d232f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:16.683691 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-c3da9f46-4ee8-4ded-98c3-af59090d232f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:16.697790 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da9f46-4ee8-4ded-98c3-af59090d232f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:16.699522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3da9f46-4ee8-4ded-98c3-af59090d232f service nova] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:16.712695 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1488/2953] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:17.430699 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd964e8-0800-44e7-ba36-c0c6f3a94f39 service placement] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:17.501746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd964e8-0800-44e7-ba36-c0c6f3a94f39 service placement] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:17.507329 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:17.508111 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:17.508497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:17.508895 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] PATH_INFO: `/v3/users/ba62460d44a043bab1f99be1897bcd74` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:17.510131 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba62460d44a043bab1f99be1897bcd74)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:17.512314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:17.533187 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:17.575425 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba62460d44a043bab1f99be1897bcd74'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:17.584942 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba62460d44a043bab1f99be1897bcd74'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:17.599083 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba62460d44a043bab1f99be1897bcd74'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:17.599837 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba62460d44a043bab1f99be1897bcd74'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:17.600478 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba62460d44a043bab1f99be1897bcd74'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:17.618319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1466/2954] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:14:17 2022] DELETE /identity/v3/users/ba62460d44a043bab1f99be1897bcd74 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:17.620082 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24449907-962f-4098-8bf4-5f10a3dc7180 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:17.620374 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24449907-962f-4098-8bf4-5f10a3dc7180 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:17.620676 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24449907-962f-4098-8bf4-5f10a3dc7180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:17.647142 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-24449907-962f-4098-8bf4-5f10a3dc7180 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:17.648532 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-24449907-962f-4098-8bf4-5f10a3dc7180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:17.669105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:17.712765 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1489/2955] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:17.723414 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:17.724906 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:17.786370 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded87c58-7966-49fe-a3ea-25009f0e01f1 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:17.790096 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88761984-6264-411f-8da6-e6923024ba8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:17.790777 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88761984-6264-411f-8da6-e6923024ba8e service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:17.791067 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88761984-6264-411f-8da6-e6923024ba8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:17.791391 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88761984-6264-411f-8da6-e6923024ba8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:17.807152 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88761984-6264-411f-8da6-e6923024ba8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:17.808745 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88761984-6264-411f-8da6-e6923024ba8e service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:17.819927 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1467/2956] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:18.101667 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24449907-962f-4098-8bf4-5f10a3dc7180 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:18.103114 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-24449907-962f-4098-8bf4-5f10a3dc7180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:18.116692 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24449907-962f-4098-8bf4-5f10a3dc7180 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:18.121839 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:18.123050 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:18.123350 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:18.123625 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] PATH_INFO: `/v3/projects/071d63d0edff418d90b93a9caabf7358` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:18.124814 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=071d63d0edff418d90b93a9caabf7358)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:18.126725 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:18.184340 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '071d63d0edff418d90b93a9caabf7358', 'request_id': 'req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.191292 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '071d63d0edff418d90b93a9caabf7358', 'request_id': 'req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.197709 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '071d63d0edff418d90b93a9caabf7358', 'request_id': 'req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.211566 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] The token cache is being invalidate because project 071d63d0edff418d90b93a9caabf7358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:14:18.211859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.212522 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.214005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1490/2957] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:14:18 2022] DELETE /identity/v3/projects/071d63d0edff418d90b93a9caabf7358 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:18.218643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88761984-6264-411f-8da6-e6923024ba8e service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:18.228421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88761984-6264-411f-8da6-e6923024ba8e service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:18.288071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88aff5-c588-4944-a450-28d477f70bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:18.288696 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88aff5-c588-4944-a450-28d477f70bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:18.288972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88aff5-c588-4944-a450-28d477f70bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:18.289242 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88aff5-c588-4944-a450-28d477f70bc1 admin admin] PATH_INFO: `/v3/users/db224a6060b2420db998d6d9c56ad9fa` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:18.290353 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c88aff5-c588-4944-a450-28d477f70bc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db224a6060b2420db998d6d9c56ad9fa)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:18.292440 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c88aff5-c588-4944-a450-28d477f70bc1 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:18.334224 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:18.340796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5c88aff5-c588-4944-a450-28d477f70bc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db224a6060b2420db998d6d9c56ad9fa'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.350508 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5c88aff5-c588-4944-a450-28d477f70bc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db224a6060b2420db998d6d9c56ad9fa'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.364698 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5c88aff5-c588-4944-a450-28d477f70bc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db224a6060b2420db998d6d9c56ad9fa'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.365566 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5c88aff5-c588-4944-a450-28d477f70bc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db224a6060b2420db998d6d9c56ad9fa'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.366167 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-5c88aff5-c588-4944-a450-28d477f70bc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db224a6060b2420db998d6d9c56ad9fa'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.382240 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1468/2958] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:14:18 2022] DELETE /identity/v3/users/db224a6060b2420db998d6d9c56ad9fa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:18.699669 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:18.701751 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:18.733089 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac40eb2-262f-4653-ae2e-bf7c2c1b3d3d admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:18.817251 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e404d-5948-40db-9ebe-30b9fb8751ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:18.818192 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e404d-5948-40db-9ebe-30b9fb8751ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:18.819101 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e404d-5948-40db-9ebe-30b9fb8751ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:18.819549 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e404d-5948-40db-9ebe-30b9fb8751ba admin admin] PATH_INFO: `/v3/projects/cb503c7069d9423a9e7a24603c62d56e` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:18.821268 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e404d-5948-40db-9ebe-30b9fb8751ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb503c7069d9423a9e7a24603c62d56e)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:18.824169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e404d-5948-40db-9ebe-30b9fb8751ba admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:18.899988 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f83e404d-5948-40db-9ebe-30b9fb8751ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb503c7069d9423a9e7a24603c62d56e', 'request_id': 'req-f83e404d-5948-40db-9ebe-30b9fb8751ba'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.901510 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:18.908153 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f83e404d-5948-40db-9ebe-30b9fb8751ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb503c7069d9423a9e7a24603c62d56e', 'request_id': 'req-f83e404d-5948-40db-9ebe-30b9fb8751ba'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.915477 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f83e404d-5948-40db-9ebe-30b9fb8751ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb503c7069d9423a9e7a24603c62d56e', 'request_id': 'req-f83e404d-5948-40db-9ebe-30b9fb8751ba'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.933856 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f83e404d-5948-40db-9ebe-30b9fb8751ba admin admin] The token cache is being invalidate because project cb503c7069d9423a9e7a24603c62d56e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81444) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:14:18.934155 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f83e404d-5948-40db-9ebe-30b9fb8751ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.940424 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-f83e404d-5948-40db-9ebe-30b9fb8751ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:18.942348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1491/2959] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:14:18 2022] DELETE /identity/v3/projects/cb503c7069d9423a9e7a24603c62d56e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:18.947228 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4e90d-2d17-4355-9c0a-fac2b01fd588 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:18.947741 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4e90d-2d17-4355-9c0a-fac2b01fd588 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:18.947868 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4e90d-2d17-4355-9c0a-fac2b01fd588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:18.969746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-72a4e90d-2d17-4355-9c0a-fac2b01fd588 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:18.970970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-72a4e90d-2d17-4355-9c0a-fac2b01fd588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:19.041019 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1469/2960] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:19.048986 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b566c9-7b8c-4b84-9e84-c3e22e3e9d4f None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.049270 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b566c9-7b8c-4b84-9e84-c3e22e3e9d4f None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.049543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b566c9-7b8c-4b84-9e84-c3e22e3e9d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.071843 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-07b566c9-7b8c-4b84-9e84-c3e22e3e9d4f None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:19.073197 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-07b566c9-7b8c-4b84-9e84-c3e22e3e9d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:19.082348 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1492/2961] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:19.086558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4e90d-2d17-4355-9c0a-fac2b01fd588 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:19.098325 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4e90d-2d17-4355-9c0a-fac2b01fd588 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:19.103319 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946034c-9148-48fd-92a2-a9098d099a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:19.103945 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946034c-9148-48fd-92a2-a9098d099a89 admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.104255 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946034c-9148-48fd-92a2-a9098d099a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.104525 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946034c-9148-48fd-92a2-a9098d099a89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.105164 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946034c-9148-48fd-92a2-a9098d099a89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81445) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:14:19.105459 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946034c-9148-48fd-92a2-a9098d099a89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:19.107237 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946034c-9148-48fd-92a2-a9098d099a89 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:19.125757 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1470/2962] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 06:14:19.127047 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b566c9-7b8c-4b84-9e84-c3e22e3e9d4f None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:19.137462 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b566c9-7b8c-4b84-9e84-c3e22e3e9d4f None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:19.142837 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33af88-ceb0-438e-b19d-526bd13c6ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:19.143441 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33af88-ceb0-438e-b19d-526bd13c6ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.143719 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33af88-ceb0-438e-b19d-526bd13c6ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.144025 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33af88-ceb0-438e-b19d-526bd13c6ca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.144713 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33af88-ceb0-438e-b19d-526bd13c6ca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:19.146619 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33af88-ceb0-438e-b19d-526bd13c6ca8 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:19.178118 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1493/2963] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:14:19 2022] POST /identity/v3/projects => generated 407 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:14:19.179178 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946034c-9148-48fd-92a2-a9098d099a89 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:19.189866 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946034c-9148-48fd-92a2-a9098d099a89 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:19.248246 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff71ff-9778-4ea0-8da9-4cc0f8c7de70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:19.248955 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff71ff-9778-4ea0-8da9-4cc0f8c7de70 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.249241 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff71ff-9778-4ea0-8da9-4cc0f8c7de70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.249512 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff71ff-9778-4ea0-8da9-4cc0f8c7de70 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.250153 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff71ff-9778-4ea0-8da9-4cc0f8c7de70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:19.252248 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff71ff-9778-4ea0-8da9-4cc0f8c7de70 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:19.274714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1471/2964] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:14:19 2022] POST /identity/v3/users => generated 304 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:14:19.278910 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33af88-ceb0-438e-b19d-526bd13c6ca8 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:19.289701 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33af88-ceb0-438e-b19d-526bd13c6ca8 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:19.295016 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcaa054-a50e-4211-8683-1148a6f89c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:19.295766 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcaa054-a50e-4211-8683-1148a6f89c77 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.296116 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcaa054-a50e-4211-8683-1148a6f89c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.296363 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcaa054-a50e-4211-8683-1148a6f89c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.297010 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcaa054-a50e-4211-8683-1148a6f89c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:19.298598 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcaa054-a50e-4211-8683-1148a6f89c77 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:19.313740 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1494/2965] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:19.318067 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff71ff-9778-4ea0-8da9-4cc0f8c7de70 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:19.328558 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff71ff-9778-4ea0-8da9-4cc0f8c7de70 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:19.333053 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb6163-cfcf-42c5-b380-bc877d45e4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:19.333640 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb6163-cfcf-42c5-b380-bc877d45e4df admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.333918 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb6163-cfcf-42c5-b380-bc877d45e4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.334184 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb6163-cfcf-42c5-b380-bc877d45e4df admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.334811 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb6163-cfcf-42c5-b380-bc877d45e4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:19.336439 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb6163-cfcf-42c5-b380-bc877d45e4df admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:19.347714 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1472/2966] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:19.351819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcaa054-a50e-4211-8683-1148a6f89c77 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:19.362478 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcaa054-a50e-4211-8683-1148a6f89c77 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:19.367087 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb752172-f816-4c91-927c-bf815c3f6f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:19.367736 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb752172-f816-4c91-927c-bf815c3f6f9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.368043 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb752172-f816-4c91-927c-bf815c3f6f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.368320 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb752172-f816-4c91-927c-bf815c3f6f9b admin admin] PATH_INFO: `/v3/projects/e4768b6a262f4f2fb08b2c20bbde7649/users/ae0d3490868f414c941b30af4c83e73a/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.384142 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb752172-f816-4c91-927c-bf815c3f6f9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4768b6a262f4f2fb08b2c20bbde7649, user_id=ae0d3490868f414c941b30af4c83e73a, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:19.386863 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb752172-f816-4c91-927c-bf815c3f6f9b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:19.400930 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1495/2967] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:14:19 2022] PUT /identity/v3/projects/e4768b6a262f4f2fb08b2c20bbde7649/users/ae0d3490868f414c941b30af4c83e73a/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:14:19.412286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45641b62-f6c3-415f-a108-729b77807e5e None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.412575 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45641b62-f6c3-415f-a108-729b77807e5e None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.412852 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45641b62-f6c3-415f-a108-729b77807e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.441673 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-45641b62-f6c3-415f-a108-729b77807e5e None None] MFA Rules not processed for user `ae0d3490868f414c941b30af4c83e73a`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:19.442874 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-45641b62-f6c3-415f-a108-729b77807e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:19.453356 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:19.510588 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1473/2968] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:19 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:19.515669 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb752172-f816-4c91-927c-bf815c3f6f9b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:19.578096 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb752172-f816-4c91-927c-bf815c3f6f9b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:19.582847 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72813de-6bb2-4ed2-af99-bbbcc32e641a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:19.583402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72813de-6bb2-4ed2-af99-bbbcc32e641a admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.583682 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72813de-6bb2-4ed2-af99-bbbcc32e641a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.583952 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72813de-6bb2-4ed2-af99-bbbcc32e641a admin admin] PATH_INFO: `/v3/projects` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.584624 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72813de-6bb2-4ed2-af99-bbbcc32e641a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:19.586539 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72813de-6bb2-4ed2-af99-bbbcc32e641a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:19.617284 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1496/2969] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:14:19 2022] POST /identity/v3/projects => generated 407 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:14:19.621338 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45641b62-f6c3-415f-a108-729b77807e5e None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:19.632334 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45641b62-f6c3-415f-a108-729b77807e5e None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:19.691427 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d98796-8027-40c6-a900-209c32b8df42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:19.692071 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d98796-8027-40c6-a900-209c32b8df42 admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.692354 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d98796-8027-40c6-a900-209c32b8df42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.692625 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d98796-8027-40c6-a900-209c32b8df42 admin admin] PATH_INFO: `/v3/users` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.693272 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d98796-8027-40c6-a900-209c32b8df42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:19.695244 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d98796-8027-40c6-a900-209c32b8df42 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:19.719513 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1474/2970] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:14:19 2022] POST /identity/v3/users => generated 310 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:14:19.723457 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72813de-6bb2-4ed2-af99-bbbcc32e641a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:19.735219 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72813de-6bb2-4ed2-af99-bbbcc32e641a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:19.740204 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbaa61-7a8c-4fd3-9960-b5d25d968a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:19.740926 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbaa61-7a8c-4fd3-9960-b5d25d968a70 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.741309 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbaa61-7a8c-4fd3-9960-b5d25d968a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.741653 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbaa61-7a8c-4fd3-9960-b5d25d968a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.742409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbaa61-7a8c-4fd3-9960-b5d25d968a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:19.744167 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbaa61-7a8c-4fd3-9960-b5d25d968a70 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:19.755629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1497/2971] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:19.760042 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d98796-8027-40c6-a900-209c32b8df42 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:19.770663 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d98796-8027-40c6-a900-209c32b8df42 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:19.775928 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4adb3-82ad-4070-82a7-33eaabe5ddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:19.776736 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4adb3-82ad-4070-82a7-33eaabe5ddf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.777017 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4adb3-82ad-4070-82a7-33eaabe5ddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.777290 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4adb3-82ad-4070-82a7-33eaabe5ddf0 admin admin] PATH_INFO: `/v3/projects/8d01d8305131436488f50afe28cb4fcc/users/d30bcf19b6e943149019e345a65f9104/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.793593 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4adb3-82ad-4070-82a7-33eaabe5ddf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d01d8305131436488f50afe28cb4fcc, user_id=d30bcf19b6e943149019e345a65f9104, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:19.796754 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4adb3-82ad-4070-82a7-33eaabe5ddf0 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:19.811619 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1475/2972] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:14:19 2022] PUT /identity/v3/projects/8d01d8305131436488f50afe28cb4fcc/users/d30bcf19b6e943149019e345a65f9104/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:19.815601 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbaa61-7a8c-4fd3-9960-b5d25d968a70 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:19.826422 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbaa61-7a8c-4fd3-9960-b5d25d968a70 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:19.873591 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:19.889449 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b1eec-2e99-4a07-9d5f-de895dc190bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:19.890097 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b1eec-2e99-4a07-9d5f-de895dc190bc admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.890386 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b1eec-2e99-4a07-9d5f-de895dc190bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.890794 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b1eec-2e99-4a07-9d5f-de895dc190bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.891446 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b1eec-2e99-4a07-9d5f-de895dc190bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:19.893264 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b1eec-2e99-4a07-9d5f-de895dc190bc admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:19.905415 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1498/2973] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:19.909172 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4adb3-82ad-4070-82a7-33eaabe5ddf0 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:19.921680 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4adb3-82ad-4070-82a7-33eaabe5ddf0 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:19.926578 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2075d-c97b-4c13-b22b-fa9950eac15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:19.927277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2075d-c97b-4c13-b22b-fa9950eac15d admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.927713 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2075d-c97b-4c13-b22b-fa9950eac15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.928028 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2075d-c97b-4c13-b22b-fa9950eac15d admin admin] PATH_INFO: `/v3/projects/8d01d8305131436488f50afe28cb4fcc/users/d30bcf19b6e943149019e345a65f9104/roles/e1f556be9f8f4d1d889117e3a132132b` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.929996 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2075d-c97b-4c13-b22b-fa9950eac15d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d01d8305131436488f50afe28cb4fcc, user_id=d30bcf19b6e943149019e345a65f9104, role_id=e1f556be9f8f4d1d889117e3a132132b)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:19.932840 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2075d-c97b-4c13-b22b-fa9950eac15d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:19.944097 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1476/2974] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:14:19 2022] PUT /identity/v3/projects/8d01d8305131436488f50afe28cb4fcc/users/d30bcf19b6e943149019e345a65f9104/roles/e1f556be9f8f4d1d889117e3a132132b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:14:19.955302 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312db3f-ebdf-4923-904e-e4b99b9dc373 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:19.955572 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312db3f-ebdf-4923-904e-e4b99b9dc373 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:19.955851 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312db3f-ebdf-4923-904e-e4b99b9dc373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:19.988797 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-b312db3f-ebdf-4923-904e-e4b99b9dc373 None None] MFA Rules not processed for user `d30bcf19b6e943149019e345a65f9104`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:19.990127 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-b312db3f-ebdf-4923-904e-e4b99b9dc373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:20.065991 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1499/2975] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:19 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:20.076817 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2075d-c97b-4c13-b22b-fa9950eac15d admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:20.078659 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-77d2075d-c97b-4c13-b22b-fa9950eac15d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:20.138128 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2075d-c97b-4c13-b22b-fa9950eac15d admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:20.141796 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13101b-a895-40fd-b133-91c6dd9f5cda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:20.142408 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13101b-a895-40fd-b133-91c6dd9f5cda service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:20.142688 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13101b-a895-40fd-b133-91c6dd9f5cda service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:20.142960 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13101b-a895-40fd-b133-91c6dd9f5cda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:20.152825 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13101b-a895-40fd-b133-91c6dd9f5cda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:20.154314 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13101b-a895-40fd-b133-91c6dd9f5cda service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:20.164161 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1477/2976] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:20 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:20.315727 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312db3f-ebdf-4923-904e-e4b99b9dc373 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:20.325223 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312db3f-ebdf-4923-904e-e4b99b9dc373 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:20.329349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad724aaa-80b8-4969-a3bf-138cda117608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:20.329938 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad724aaa-80b8-4969-a3bf-138cda117608 service cinder] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:20.330213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad724aaa-80b8-4969-a3bf-138cda117608 service cinder] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:20.330481 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad724aaa-80b8-4969-a3bf-138cda117608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:20.393289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad724aaa-80b8-4969-a3bf-138cda117608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:20.394641 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad724aaa-80b8-4969-a3bf-138cda117608 service cinder] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:20.404145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1500/2977] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:20 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:20.692782 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13101b-a895-40fd-b133-91c6dd9f5cda service cinder] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:20.694195 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-3e13101b-a895-40fd-b133-91c6dd9f5cda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:20.707975 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:20.786514 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13101b-a895-40fd-b133-91c6dd9f5cda service cinder] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:20.791778 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c3e3b-c3b5-4e65-8080-15ebac82e672 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:20.792451 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c3e3b-c3b5-4e65-8080-15ebac82e672 service placement] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:20.792751 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c3e3b-c3b5-4e65-8080-15ebac82e672 service placement] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:20.793049 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c3e3b-c3b5-4e65-8080-15ebac82e672 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:20.796085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-729c3e3b-c3b5-4e65-8080-15ebac82e672 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:20.809214 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c3e3b-c3b5-4e65-8080-15ebac82e672 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:20.811114 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c3e3b-c3b5-4e65-8080-15ebac82e672 service placement] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:20.821605 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1478/2978] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:14:29.896811 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad724aaa-80b8-4969-a3bf-138cda117608 service cinder] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:29.963296 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad724aaa-80b8-4969-a3bf-138cda117608 service cinder] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:29.968543 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:29.969174 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:29.969583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:29.969857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] PATH_INFO: `/v3/users/ae0d3490868f414c941b30af4c83e73a` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:29.971003 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0d3490868f414c941b30af4c83e73a)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:29.972965 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:29.983694 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=100 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:30.022547 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0d3490868f414c941b30af4c83e73a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.023290 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0d3490868f414c941b30af4c83e73a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.030764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0d3490868f414c941b30af4c83e73a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.038514 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0d3490868f414c941b30af4c83e73a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.052066 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0d3490868f414c941b30af4c83e73a'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.058293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1501/2979] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:14:29 2022] DELETE /identity/v3/users/ae0d3490868f414c941b30af4c83e73a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:14:30.062784 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d8f08-af32-4336-b3ae-b88e5c696314 None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:30.063266 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d8f08-af32-4336-b3ae-b88e5c696314 None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:30.063380 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d8f08-af32-4336-b3ae-b88e5c696314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:30.082676 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-8d1d8f08-af32-4336-b3ae-b88e5c696314 None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:30.083865 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8d1d8f08-af32-4336-b3ae-b88e5c696314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:30.144061 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1479/2980] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:30.154752 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:30.156209 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:30.214002 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e4464-9414-4585-b454-b5411641f42a admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:30.217656 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc33d9-06af-4012-b930-a38a4bf45cec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:30.218261 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc33d9-06af-4012-b930-a38a4bf45cec service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:30.218540 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc33d9-06af-4012-b930-a38a4bf45cec service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:30.218812 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc33d9-06af-4012-b930-a38a4bf45cec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:30.228169 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc33d9-06af-4012-b930-a38a4bf45cec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:30.229602 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc33d9-06af-4012-b930-a38a4bf45cec service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:30.239678 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1502/2981] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:30.531014 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d8f08-af32-4336-b3ae-b88e5c696314 None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:30.532480 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-8d1d8f08-af32-4336-b3ae-b88e5c696314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:30.552286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d8f08-af32-4336-b3ae-b88e5c696314 None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:30.574298 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:30.574952 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:30.575277 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:30.575559 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] PATH_INFO: `/v3/projects/e4768b6a262f4f2fb08b2c20bbde7649` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:30.576715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4768b6a262f4f2fb08b2c20bbde7649)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:30.578638 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:30.670670 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4768b6a262f4f2fb08b2c20bbde7649', 'request_id': 'req-c5a78b72-7739-4fe4-bb5d-60255db32bcf'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.683655 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4768b6a262f4f2fb08b2c20bbde7649', 'request_id': 'req-c5a78b72-7739-4fe4-bb5d-60255db32bcf'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.691970 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4768b6a262f4f2fb08b2c20bbde7649', 'request_id': 'req-c5a78b72-7739-4fe4-bb5d-60255db32bcf'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.710502 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] The token cache is being invalidate because project e4768b6a262f4f2fb08b2c20bbde7649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:14:30.710996 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.712394 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.717743 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1480/2982] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:14:30 2022] DELETE /identity/v3/projects/e4768b6a262f4f2fb08b2c20bbde7649 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:30.725862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc33d9-06af-4012-b930-a38a4bf45cec service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:30.746349 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc33d9-06af-4012-b930-a38a4bf45cec service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:30.817113 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af877d2-1ec8-4b8e-a7a9-2f15a766fae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:30.817747 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af877d2-1ec8-4b8e-a7a9-2f15a766fae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:30.818021 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af877d2-1ec8-4b8e-a7a9-2f15a766fae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:30.818301 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af877d2-1ec8-4b8e-a7a9-2f15a766fae9 admin admin] PATH_INFO: `/v3/users/d30bcf19b6e943149019e345a65f9104` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:30.819427 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af877d2-1ec8-4b8e-a7a9-2f15a766fae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d30bcf19b6e943149019e345a65f9104)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:30.821381 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af877d2-1ec8-4b8e-a7a9-2f15a766fae9 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:30.858183 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:30.872707 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1af877d2-1ec8-4b8e-a7a9-2f15a766fae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd30bcf19b6e943149019e345a65f9104'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.873541 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1af877d2-1ec8-4b8e-a7a9-2f15a766fae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd30bcf19b6e943149019e345a65f9104'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.880469 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1af877d2-1ec8-4b8e-a7a9-2f15a766fae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd30bcf19b6e943149019e345a65f9104'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.889809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1af877d2-1ec8-4b8e-a7a9-2f15a766fae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd30bcf19b6e943149019e345a65f9104'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.902642 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-1af877d2-1ec8-4b8e-a7a9-2f15a766fae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd30bcf19b6e943149019e345a65f9104'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:30.909428 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1503/2983] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:14:30 2022] DELETE /identity/v3/users/d30bcf19b6e943149019e345a65f9104 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:31.185688 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:31.186270 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:31.199468 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a78b72-7739-4fe4-bb5d-60255db32bcf admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:31.219766 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:31.266176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1165d-16fb-46fe-8fea-e535b1e17636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:31.266715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1165d-16fb-46fe-8fea-e535b1e17636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:31.266990 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1165d-16fb-46fe-8fea-e535b1e17636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:31.267256 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1165d-16fb-46fe-8fea-e535b1e17636 admin admin] PATH_INFO: `/v3/projects/8d01d8305131436488f50afe28cb4fcc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:31.268402 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1165d-16fb-46fe-8fea-e535b1e17636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d01d8305131436488f50afe28cb4fcc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:31.270302 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1165d-16fb-46fe-8fea-e535b1e17636 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:31.318196 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-67b1165d-16fb-46fe-8fea-e535b1e17636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d01d8305131436488f50afe28cb4fcc', 'request_id': 'req-67b1165d-16fb-46fe-8fea-e535b1e17636'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:31.325623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-67b1165d-16fb-46fe-8fea-e535b1e17636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d01d8305131436488f50afe28cb4fcc', 'request_id': 'req-67b1165d-16fb-46fe-8fea-e535b1e17636'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:31.332477 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-67b1165d-16fb-46fe-8fea-e535b1e17636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d01d8305131436488f50afe28cb4fcc', 'request_id': 'req-67b1165d-16fb-46fe-8fea-e535b1e17636'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:31.343142 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-67b1165d-16fb-46fe-8fea-e535b1e17636 admin admin] The token cache is being invalidate because project 8d01d8305131436488f50afe28cb4fcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:14:31.343439 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-67b1165d-16fb-46fe-8fea-e535b1e17636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:31.344155 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-67b1165d-16fb-46fe-8fea-e535b1e17636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:31.345631 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1481/2984] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:14:31 2022] DELETE /identity/v3/projects/8d01d8305131436488f50afe28cb4fcc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:31.352575 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342464c-65ca-4b94-8232-c415e4c13a1c None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:31.352862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342464c-65ca-4b94-8232-c415e4c13a1c None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:31.353134 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342464c-65ca-4b94-8232-c415e4c13a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:31.374644 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-0342464c-65ca-4b94-8232-c415e4c13a1c None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:31.375846 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-0342464c-65ca-4b94-8232-c415e4c13a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:31.437148 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1504/2985] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:14:31.445108 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926a9af-e871-4a15-9432-085ea0c3ab5d None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:31.445527 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926a9af-e871-4a15-9432-085ea0c3ab5d None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:31.445813 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926a9af-e871-4a15-9432-085ea0c3ab5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:31.466065 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-5926a9af-e871-4a15-9432-085ea0c3ab5d None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:31.467346 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-5926a9af-e871-4a15-9432-085ea0c3ab5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:31.475423 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1482/2986] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 06:14:31.479628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342464c-65ca-4b94-8232-c415e4c13a1c None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:31.491372 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342464c-65ca-4b94-8232-c415e4c13a1c None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:31.495959 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47ae50-3471-483e-b01d-2a63f17b7f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:31.496586 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47ae50-3471-483e-b01d-2a63f17b7f34 admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:31.496862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47ae50-3471-483e-b01d-2a63f17b7f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:31.497857 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47ae50-3471-483e-b01d-2a63f17b7f34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:31.498836 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47ae50-3471-483e-b01d-2a63f17b7f34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81444) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 06:14:31.499137 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47ae50-3471-483e-b01d-2a63f17b7f34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:31.500768 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47ae50-3471-483e-b01d-2a63f17b7f34 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:31.515831 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1505/2987] 10.222.0.33 () {58 vars in 1153 bytes} [Sun Sep 11 06:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 06:14:31.516959 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926a9af-e871-4a15-9432-085ea0c3ab5d None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:31.526465 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926a9af-e871-4a15-9432-085ea0c3ab5d None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:31.531201 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9c507-4916-4cc4-9cae-3011fee5624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:31.531873 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9c507-4916-4cc4-9cae-3011fee5624c admin admin] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:31.532268 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9c507-4916-4cc4-9cae-3011fee5624c admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:31.532610 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9c507-4916-4cc4-9cae-3011fee5624c admin admin] PATH_INFO: `/v3/projects` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:31.533279 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9c507-4916-4cc4-9cae-3011fee5624c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:31.535192 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9c507-4916-4cc4-9cae-3011fee5624c admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:31.563634 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1483/2988] 10.222.0.33 () {60 vars in 1153 bytes} [Sun Sep 11 06:14:31 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 06:14:31.568073 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47ae50-3471-483e-b01d-2a63f17b7f34 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:31.578335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47ae50-3471-483e-b01d-2a63f17b7f34 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:31.636828 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39738e5d-3227-4a21-bdf5-d75938f4a6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:31.637435 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39738e5d-3227-4a21-bdf5-d75938f4a6d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:31.638293 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39738e5d-3227-4a21-bdf5-d75938f4a6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:31.638716 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39738e5d-3227-4a21-bdf5-d75938f4a6d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:31.639342 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39738e5d-3227-4a21-bdf5-d75938f4a6d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:31.641321 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39738e5d-3227-4a21-bdf5-d75938f4a6d6 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:31.663583 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1506/2989] 10.222.0.33 () {60 vars in 1144 bytes} [Sun Sep 11 06:14:31 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 06:14:31.667753 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9c507-4916-4cc4-9cae-3011fee5624c admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:31.678315 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9c507-4916-4cc4-9cae-3011fee5624c admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:31.687050 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53716861-ee14-4134-b8a4-ef7c3c00e26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:31.687656 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53716861-ee14-4134-b8a4-ef7c3c00e26a admin admin] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:31.687934 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53716861-ee14-4134-b8a4-ef7c3c00e26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:31.688233 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53716861-ee14-4134-b8a4-ef7c3c00e26a admin admin] PATH_INFO: `/v3/roles` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:31.688871 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53716861-ee14-4134-b8a4-ef7c3c00e26a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:31.690497 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53716861-ee14-4134-b8a4-ef7c3c00e26a admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:31.701296 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1484/2990] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:31.705322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39738e5d-3227-4a21-bdf5-d75938f4a6d6 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:31.715244 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39738e5d-3227-4a21-bdf5-d75938f4a6d6 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:31.720402 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6904a0-9cd9-413b-995f-99cf888216dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:31.721018 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6904a0-9cd9-413b-995f-99cf888216dc admin admin] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:31.721295 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6904a0-9cd9-413b-995f-99cf888216dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:31.721564 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6904a0-9cd9-413b-995f-99cf888216dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:31.722193 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6904a0-9cd9-413b-995f-99cf888216dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:31.723773 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6904a0-9cd9-413b-995f-99cf888216dc admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:31.734565 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1507/2991] 10.222.0.33 () {58 vars in 1122 bytes} [Sun Sep 11 06:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 06:14:31.738834 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53716861-ee14-4134-b8a4-ef7c3c00e26a admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:31.749920 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53716861-ee14-4134-b8a4-ef7c3c00e26a admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:31.754937 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ce4b5-90d4-41af-857d-f2b69238d4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:31.755606 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ce4b5-90d4-41af-857d-f2b69238d4a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:31.755884 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ce4b5-90d4-41af-857d-f2b69238d4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:31.756295 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ce4b5-90d4-41af-857d-f2b69238d4a8 admin admin] PATH_INFO: `/v3/projects/53849ceaa0634f899a5f14ac7b0ec269/users/ef55befaacf6420d87b756496d23f8d1/roles/d9f6babaca2d494c9a04f0aca1271af8` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:31.771681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ce4b5-90d4-41af-857d-f2b69238d4a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53849ceaa0634f899a5f14ac7b0ec269, user_id=ef55befaacf6420d87b756496d23f8d1, role_id=d9f6babaca2d494c9a04f0aca1271af8)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:31.774574 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ce4b5-90d4-41af-857d-f2b69238d4a8 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:31.788791 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1485/2992] 10.222.0.33 () {60 vars in 1483 bytes} [Sun Sep 11 06:14:31 2022] PUT /identity/v3/projects/53849ceaa0634f899a5f14ac7b0ec269/users/ef55befaacf6420d87b756496d23f8d1/roles/d9f6babaca2d494c9a04f0aca1271af8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:14:31.801088 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4acad2-a6d8-4392-8684-02b7a5e5a1c6 None None] REQUEST_METHOD: `POST` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:31.801393 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4acad2-a6d8-4392-8684-02b7a5e5a1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:31.801659 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4acad2-a6d8-4392-8684-02b7a5e5a1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:31.831876 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.auth.core [None req-dc4acad2-a6d8-4392-8684-02b7a5e5a1c6 None None] MFA Rules not processed for user `ef55befaacf6420d87b756496d23f8d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81444) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:31.833124 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-dc4acad2-a6d8-4392-8684-02b7a5e5a1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:31.837115 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:31.903208 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1508/2993] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:31 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:14:31.915085 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ce4b5-90d4-41af-857d-f2b69238d4a8 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:31.916809 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-2b4ce4b5-90d4-41af-857d-f2b69238d4a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:31.969972 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:31.976409 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ce4b5-90d4-41af-857d-f2b69238d4a8 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:31.980953 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fe804-7c22-4661-b0cb-e52dd0446601 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ef93155a40f4c69b320ca962d4e8868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:31.982600 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fe804-7c22-4661-b0cb-e52dd0446601 service cinder] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:31.983146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fe804-7c22-4661-b0cb-e52dd0446601 service cinder] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:31.983955 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fe804-7c22-4661-b0cb-e52dd0446601 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:31.993234 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fe804-7c22-4661-b0cb-e52dd0446601 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:31.994658 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fe804-7c22-4661-b0cb-e52dd0446601 service cinder] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:32.004371 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1486/2994] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:31 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:41.167688 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4acad2-a6d8-4392-8684-02b7a5e5a1c6 None None] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:41.235013 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4acad2-a6d8-4392-8684-02b7a5e5a1c6 None None] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:41.239562 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:41.240227 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:41.240537 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:41.240809 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] PATH_INFO: `/v3/users/ef55befaacf6420d87b756496d23f8d1` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:41.241939 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef55befaacf6420d87b756496d23f8d1)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:41.243859 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:41.292819 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef55befaacf6420d87b756496d23f8d1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:41.293484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef55befaacf6420d87b756496d23f8d1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:41.300763 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef55befaacf6420d87b756496d23f8d1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:41.308195 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef55befaacf6420d87b756496d23f8d1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:41.321484 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef55befaacf6420d87b756496d23f8d1'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:41.328304 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1509/2995] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:14:41 2022] DELETE /identity/v3/users/ef55befaacf6420d87b756496d23f8d1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:41.337087 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e753c26-d32a-4a2b-ad2c-ad05e05ab0bc None None] REQUEST_METHOD: `POST` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:41.337581 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e753c26-d32a-4a2b-ad2c-ad05e05ab0bc None None] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:41.337710 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e753c26-d32a-4a2b-ad2c-ad05e05ab0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:41.357855 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.auth.core [None req-7e753c26-d32a-4a2b-ad2c-ad05e05ab0bc None None] MFA Rules not processed for user `f615cddbb3154c329188ec780374eff5`. Rule list: `[]` (Enabled: `True`). {{(pid=81445) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 06:14:41.359093 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7e753c26-d32a-4a2b-ad2c-ad05e05ab0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:41.420679 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1487/2996] 10.222.0.33 () {58 vars in 958 bytes} [Sun Sep 11 06:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 06:14:41.430288 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:41.431731 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:41.460825 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:41.492623 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e3cd8-d38e-466e-b0d4-2d0558d43c9b admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:41.496413 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98fa76-c27e-4a93-882b-f6f3cc8bb00b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:41.497686 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98fa76-c27e-4a93-882b-f6f3cc8bb00b service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:41.498058 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98fa76-c27e-4a93-882b-f6f3cc8bb00b service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:41.498338 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98fa76-c27e-4a93-882b-f6f3cc8bb00b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:41.507499 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98fa76-c27e-4a93-882b-f6f3cc8bb00b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:41.508961 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98fa76-c27e-4a93-882b-f6f3cc8bb00b service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:41.519006 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1510/2997] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:14:41.793169 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e753c26-d32a-4a2b-ad2c-ad05e05ab0bc None None] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:14:41.795122 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-7e753c26-d32a-4a2b-ad2c-ad05e05ab0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:14:41.811211 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e753c26-d32a-4a2b-ad2c-ad05e05ab0bc None None] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:14:41.818103 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:14:41.818908 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:14:41.819191 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:14:41.819461 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] PATH_INFO: `/v3/projects/53849ceaa0634f899a5f14ac7b0ec269` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:14:41.820675 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53849ceaa0634f899a5f14ac7b0ec269)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:14:41.822613 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:14:41.876081 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53849ceaa0634f899a5f14ac7b0ec269', 'request_id': 'req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:41.883719 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53849ceaa0634f899a5f14ac7b0ec269', 'request_id': 'req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:41.891182 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53849ceaa0634f899a5f14ac7b0ec269', 'request_id': 'req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:41.902267 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] The token cache is being invalidate because project 53849ceaa0634f899a5f14ac7b0ec269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:14:41.902636 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:41.904015 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:14:41.905650 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1488/2998] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:14:41 2022] DELETE /identity/v3/projects/53849ceaa0634f899a5f14ac7b0ec269 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:14:51.514658 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=15 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:14:51.899176 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:16:07.048327 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98fa76-c27e-4a93-882b-f6f3cc8bb00b service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:16:07.049780 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-2e98fa76-c27e-4a93-882b-f6f3cc8bb00b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:16:07.063764 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98fa76-c27e-4a93-882b-f6f3cc8bb00b service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:16:07.112409 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b44f9a-b520-4c16-97d7-79f2dba28dcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:16:07.113024 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b44f9a-b520-4c16-97d7-79f2dba28dcf service neutron] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:16:07.113303 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b44f9a-b520-4c16-97d7-79f2dba28dcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:16:07.113576 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b44f9a-b520-4c16-97d7-79f2dba28dcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:16:07.115326 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-64b44f9a-b520-4c16-97d7-79f2dba28dcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:16:07.174485 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b44f9a-b520-4c16-97d7-79f2dba28dcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:16:07.175946 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b44f9a-b520-4c16-97d7-79f2dba28dcf service neutron] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:16:07.185472 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1511/2999] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:16:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:16:13.468337 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:16:13.469515 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:16:13.506504 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023fe87-8e67-4c5b-bdda-2892e93a0ad9 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:16:13.509998 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d625ee-15b1-4b98-806b-ff46254e7c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bf2a5de0e744505a1317ce9fa224643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:16:13.510602 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d625ee-15b1-4b98-806b-ff46254e7c52 service neutron] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:16:13.510878 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d625ee-15b1-4b98-806b-ff46254e7c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:16:13.511144 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d625ee-15b1-4b98-806b-ff46254e7c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:16:13.512902 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-c8d625ee-15b1-4b98-806b-ff46254e7c52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:16:13.581807 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d625ee-15b1-4b98-806b-ff46254e7c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:16:13.583260 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d625ee-15b1-4b98-806b-ff46254e7c52 service neutron] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:16:13.593730 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1489/3000] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:16:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 06:16:16.829339 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b44f9a-b520-4c16-97d7-79f2dba28dcf service neutron] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:16:16.831341 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-64b44f9a-b520-4c16-97d7-79f2dba28dcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:16:16.933942 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b44f9a-b520-4c16-97d7-79f2dba28dcf service neutron] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:16:16.940375 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19425433-fa3c-413f-af32-ac85366e0afa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:16:16.941002 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425433-fa3c-413f-af32-ac85366e0afa service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:16:16.941350 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425433-fa3c-413f-af32-ac85366e0afa service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:16:16.941628 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425433-fa3c-413f-af32-ac85366e0afa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:16:16.952029 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-19425433-fa3c-413f-af32-ac85366e0afa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:16:16.958429 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19425433-fa3c-413f-af32-ac85366e0afa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:16:16.960073 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19425433-fa3c-413f-af32-ac85366e0afa service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:16:16.974021 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1512/3001] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:16:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:16:20.173551 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d625ee-15b1-4b98-806b-ff46254e7c52 service neutron] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:16:20.198683 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d625ee-15b1-4b98-806b-ff46254e7c52 service neutron] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:16:20.211884 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5081d-2af9-4ff6-9f2b-99cb6ca596d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:16:20.212531 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5081d-2af9-4ff6-9f2b-99cb6ca596d5 service placement] REQUEST_METHOD: `GET` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:16:20.213012 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5081d-2af9-4ff6-9f2b-99cb6ca596d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:16:20.213286 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5081d-2af9-4ff6-9f2b-99cb6ca596d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:16:20.215087 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-b8e5081d-2af9-4ff6-9f2b-99cb6ca596d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:16:20.229579 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5081d-2af9-4ff6-9f2b-99cb6ca596d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:16:20.231438 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5081d-2af9-4ff6-9f2b-99cb6ca596d5 service placement] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:16:20.246750 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19425433-fa3c-413f-af32-ac85366e0afa service placement] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:16:20.250262 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-19425433-fa3c-413f-af32-ac85366e0afa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:16:20.256715 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1490/3002] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:16:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:16:20.282004 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19425433-fa3c-413f-af32-ac85366e0afa service placement] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:16:20.287796 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:16:20.288466 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:16:20.289862 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:16:20.290322 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] PATH_INFO: `/v3/users/309d1efd676845bbaa30f472b19ffa34` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:16:20.291703 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=309d1efd676845bbaa30f472b19ffa34)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:16:20.293850 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:16:20.362977 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:16:20.373369 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '309d1efd676845bbaa30f472b19ffa34'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.374176 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '309d1efd676845bbaa30f472b19ffa34'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.382246 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '309d1efd676845bbaa30f472b19ffa34'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.391742 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '309d1efd676845bbaa30f472b19ffa34'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.406399 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '309d1efd676845bbaa30f472b19ffa34'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.416459 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1513/3003] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:16:20 2022] DELETE /identity/v3/users/309d1efd676845bbaa30f472b19ffa34 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 06:16:20.589232 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5081d-2af9-4ff6-9f2b-99cb6ca596d5 service placement] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:16:20.600637 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5081d-2af9-4ff6-9f2b-99cb6ca596d5 service placement] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:16:20.675120 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:16:20.675758 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:16:20.676062 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:16:20.676333 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] PATH_INFO: `/v3/projects/10427ce4a1644db89943273919d296cc` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:16:20.677494 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10427ce4a1644db89943273919d296cc)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:16:20.679421 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:16:20.730070 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10427ce4a1644db89943273919d296cc', 'request_id': 'req-4166cace-aa16-4039-91df-c7f6a6168094'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.739583 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10427ce4a1644db89943273919d296cc', 'request_id': 'req-4166cace-aa16-4039-91df-c7f6a6168094'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.746235 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10427ce4a1644db89943273919d296cc', 'request_id': 'req-4166cace-aa16-4039-91df-c7f6a6168094'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.753643 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:16:20.757806 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] The token cache is being invalidate because project 10427ce4a1644db89943273919d296cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:16:20.758284 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.759278 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.761146 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1491/3004] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:16:20 2022] DELETE /identity/v3/projects/10427ce4a1644db89943273919d296cc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:16:20.765607 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:16:20.767335 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:16:20.783060 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c4751-576c-4a35-9c48-16477f8c9375 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:16:20.850629 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:16:20.851289 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:16:20.851570 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:16:20.851841 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] PATH_INFO: `/v3/users/2bae12693aac49a2b63d13cf23cfbd3d` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:16:20.853037 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bae12693aac49a2b63d13cf23cfbd3d)` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:16:20.854976 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:16:20.905589 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bae12693aac49a2b63d13cf23cfbd3d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.906243 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bae12693aac49a2b63d13cf23cfbd3d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.913413 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bae12693aac49a2b63d13cf23cfbd3d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.922213 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bae12693aac49a2b63d13cf23cfbd3d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.934370 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.notifications [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bae12693aac49a2b63d13cf23cfbd3d'} {{(pid=81444) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:20.941105 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1514/3005] 10.222.0.33 () {58 vars in 1224 bytes} [Sun Sep 11 06:16:20 2022] DELETE /identity/v3/users/2bae12693aac49a2b63d13cf23cfbd3d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:16:21.122963 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] Authenticating user token {{(pid=81445) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:16:21.124746 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.fernet_utils [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81445) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:16:21.141623 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166cace-aa16-4039-91df-c7f6a6168094 admin admin] Validating token access rules against request {{(pid=81445) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:16:21.205027 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32de9a-9825-46f4-bf9b-4c32fa23306d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f615cddbb3154c329188ec780374eff5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c05020216314cd9bc11af667152fcc9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81445) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:16:21.205720 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32de9a-9825-46f4-bf9b-4c32fa23306d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:16:21.205936 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32de9a-9825-46f4-bf9b-4c32fa23306d admin admin] SCRIPT_NAME: `/identity` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:16:21.206347 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32de9a-9825-46f4-bf9b-4c32fa23306d admin admin] PATH_INFO: `/v3/projects/6c69561786614f9ebb42a36816416489` {{(pid=81445) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:16:21.207534 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32de9a-9825-46f4-bf9b-4c32fa23306d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c69561786614f9ebb42a36816416489)` {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:16:21.209534 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32de9a-9825-46f4-bf9b-4c32fa23306d admin admin] RBAC: Authorization granted {{(pid=81445) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:16:21.257798 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4a32de9a-9825-46f4-bf9b-4c32fa23306d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c69561786614f9ebb42a36816416489', 'request_id': 'req-4a32de9a-9825-46f4-bf9b-4c32fa23306d'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:21.265889 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4a32de9a-9825-46f4-bf9b-4c32fa23306d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c69561786614f9ebb42a36816416489', 'request_id': 'req-4a32de9a-9825-46f4-bf9b-4c32fa23306d'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:21.272386 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4a32de9a-9825-46f4-bf9b-4c32fa23306d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c69561786614f9ebb42a36816416489', 'request_id': 'req-4a32de9a-9825-46f4-bf9b-4c32fa23306d'} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:21.284223 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4a32de9a-9825-46f4-bf9b-4c32fa23306d admin admin] The token cache is being invalidate because project 6c69561786614f9ebb42a36816416489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81445) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 06:16:21.284520 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4a32de9a-9825-46f4-bf9b-4c32fa23306d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:21.285192 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG keystone.notifications [None req-4a32de9a-9825-46f4-bf9b-4c32fa23306d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81445) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 06:16:21.286681 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: [pid: 81445|app: 0|req: 1492/3006] 10.222.0.33 () {58 vars in 1233 bytes} [Sun Sep 11 06:16:21 2022] DELETE /identity/v3/projects/6c69561786614f9ebb42a36816416489 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 06:16:30.938236 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:DELETE=15,keystone:SELECT=67,keystone:INSERT=2 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:16:31.281309 dv-cinder-856921-1-iscsi devstack@keystone.service[81445]: DEBUG dbcounter [-] [81445] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81445) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 06:19:26.683005 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] Authenticating user token {{(pid=81444) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 06:19:26.684585 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:19:26.802419 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85524892-b7fe-4ba5-a95a-15a1329283b3 admin admin] Validating token access rules against request {{(pid=81444) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 06:19:26.863672 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd66e72-089c-4e35-9aff-e3a792e69d7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e985df8ad6c4bc7b3cd1328924e3ce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '092dd30ff8b541e2a45947504831998a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81444) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 06:19:26.864332 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd66e72-089c-4e35-9aff-e3a792e69d7a service placement] REQUEST_METHOD: `GET` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 06:19:26.864761 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd66e72-089c-4e35-9aff-e3a792e69d7a service placement] SCRIPT_NAME: `/identity` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 06:19:26.865145 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd66e72-089c-4e35-9aff-e3a792e69d7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81444) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 06:19:26.866869 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.fernet_utils [None req-5bd66e72-089c-4e35-9aff-e3a792e69d7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81444) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 06:19:26.878330 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd66e72-089c-4e35-9aff-e3a792e69d7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 06:19:26.879955 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd66e72-089c-4e35-9aff-e3a792e69d7a service placement] RBAC: Authorization granted {{(pid=81444) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 06:19:26.890759 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: [pid: 81444|app: 0|req: 1515/3007] 10.222.0.33 () {60 vars in 1365 bytes} [Sun Sep 11 06:19:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 06:19:36.885714 dv-cinder-856921-1-iscsi devstack@keystone.service[81444]: DEBUG dbcounter [-] [81444] Writing DB stats keystone:SELECT=75 {{(pid=81444) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}